Stop shipping “AI features.” Ship delegation.
If your AI story still ends at “it writes a good answer,” you’re behind. The market already normalized chat: ChatGPT pulled mainstream usage into the hundreds of millions, and Microsoft, Google, and Salesforce pushed assistants into their core suites. That made “ask the product” ordinary. The next expectation is blunt: the product should do the work.
That expectation breaks most B2B apps because the real workflow still lives behind buttons, permissions, brittle integrations, and messy data. A model can draft a perfect email, but the customer still has to open five tabs, reconcile fields, attach a document, and hit send. That gap is where agent-ready products win: they let software take actions across tools—under tight controls.
Agent-ready doesn’t mean you let an LLM loose in production. It means you design for delegated execution: reliable tool APIs, explicit permissions, auditable actions, cost ceilings, and user-visible state. It means the architecture assumes the “actor” might be an agent, not a person clicking around. The moat is operational: contracts, controls, and trust.
The teams that get this stop treating LLMs like a shiny UI layer. They treat them like a new runtime that needs budgets, observability, rollback, and clear failure behavior. You don’t “add agents.” You define what they’re allowed to do, make tools predictable, and make mistakes cheap to find and undo.
Design around the delegation loop, not the click loop
Classic SaaS assumes a click loop: intent → click → response → next click. Agents flip that into a delegation loop: intent → plan → tool calls → verification → escalation. Your product either makes that loop visible and controllable, or users (and security teams) refuse to trust it.
In agent UX, “trust” is mostly visibility. Show what the agent is doing, what it plans to do next, which systems it touched, and what’s pending. Notion, Atlassian, and Microsoft all keep circling the same truth with their AI directions: people delegate when they can see state and review outputs in context—not when the app just produces fluent text.
Pattern: plan first, execute second
Separate planning from execution. Make the agent present a concrete checklist of steps, then gate the steps that carry risk. If a step is irreversible (sending an email, issuing a refund, changing production config), require an explicit approval. If it’s reversible or low impact (tagging, routing, summarizing), it can run with a clear notification and a trail.
Pattern: review surfaces beat chat transcripts
A chat history is not supervision. Review surfaces are supervision: diffs, timelines, change summaries, and “what changed” views. GitHub’s pull request is a useful mental model: the winning UX isn’t just writing—it's reviewing and merging safely. Agent-ready apps need the equivalent for documents, CRM fields, tickets, billing records, and workflows.
Make three product affordances non-optional: (1) an approval gate for irreversible actions, (2) verification for actions you can check automatically (matching IDs, totals, policy rules), and (3) a fast escalation path when confidence drops or data is missing. Most “AI features” stall here because teams treat it like prompt tuning instead of product design.
Table 1: Common agent UX patterns (what they optimize for and where they usually break)
| Pattern | Best for | Typical KPI impact | Common failure mode |
|---|---|---|---|
| Propose → Approve → Execute | High-risk actions (payments, outbound messages, policy decisions) | Higher completion with fewer incidents | Approval friction turns into a bottleneck |
| Autopilot with Notifications | Low-risk ops (tagging, routing, summaries, enrichment) | Higher throughput; shorter cycle time | Silent mistakes erode trust |
| Human-in-the-Loop Queue | Support, trust & safety, compliance review | More consistency; less rework | Queue backlog if thresholds are too conservative |
| Diff-first Review Surface | Docs, code, configs, structured records | Faster approvals; fewer “mystery changes” | Bad diffs hide meaning-changing edits |
| Tool-only Mode (no free text) | Regulated workflows; deterministic execution | Lower variance; simpler audits | Feels rigid when users need flexibility |
Prompts don’t scale. Systems do.
Prompting mattered—until you tried to run it in production. Agents fail for boring reasons: missing state, inconsistent schemas, unreliable search, permission mismatches, and tools that error in ways humans can intuit but software can’t. That’s why “agent performance” is mostly a product and engineering systems problem.
Three building blocks decide whether an agent behaves: durable state, tools, and constraints. Durable state means your product stores real memory: preferences, entity resolution, permissions, task history, and what already happened. Tools mean stable function interfaces for read, write, and side effects. Constraints mean hard limits on time, tool calls, cost, and allowed actions so the system fails cleanly instead of spiraling.
Tool quality is where most apps lose. If your create/update endpoints change shape across tenants, the agent “looks flaky” even if the model is fine. If your search returns duplicates and junk, the agent will pull the wrong document and act confidently. Treat tool behavior like a product surface: version it, document it, monitor it, and make failures explicit. If you already run an internal developer platform mindset—stable interfaces, observability, clear error handling—you’re ahead.
Constraints are the other half of seriousness. Put ceilings on work: a maximum number of tool calls, a wall-time limit, a spend cap, and a deterministic rule for escalation. Without those, an agent will burn time and money chasing completion. With them, it behaves like a system you can operate.
“You can’t just run a model; you have to run a whole system.” — Dario Amodei, Anthropic (public interviews and talks)
Instrumentation is the UX: measure autonomy like production reliability
Funnels and cohorts don’t tell you if an agent is safe. Agent analytics needs an ops spine: how often tasks complete without a human stepping in, how often users edit or stop actions, what incidents occur, and what each outcome costs. If you can’t answer those questions, you’re shipping vibes.
The metrics that matter are consistent across most products:
- Autonomy rate: tasks completed end-to-end without escalation.
- Intervention rate: how often people edit, override, or cancel.
- Incident rate: failures per task, split by severity.
- Cost per resolved task: full stack cost tied to completed outcomes.
The task ledger
A task ledger is the simplest pattern that fixes multiple problems at once. Log each task with: the plan, tool calls, inputs/outputs, approvals, diffs, and final outcome—plus correlation IDs across the model and the tools. That gives you auditability, debuggability, and cost allocation. It also makes governance and procurement conversations concrete because you can show what actually happened, not what a demo implied.
Eval like production: replay real work
Offline benchmarks don’t reflect your permissions model, your data quality, or your tool failures. The clean approach is replay evaluation: re-run a corpus of real tasks against a new model or policy and compare outcomes, costs, and incidents before you ship. If it breaks replay, it breaks production—don’t ship it.
Measure outcomes, not eloquence. Nobody buys “good reasoning.” They buy closed tickets, reconciled invoices, updated records, and messages that were actually sent correctly.
Security in agent products is blast-radius design
Enterprise buyers aren’t debating whether LLMs exist. They’re standardizing how AI is allowed to operate. The failure mode they fear isn’t “hallucinations” as an abstract concept; it’s an agent taking a wrong action at machine speed across systems.
Start with permissions that match how businesses actually work. If your agent is just impersonating a user account, you’re setting yourself up for ugly edge cases. Mature designs use scoped identities (service principals) with time-bound access and explicit separation between read and act, draft and send, propose and commit. That separation is the difference between “helpful assistant” and “automated incident generator.”
Next: auditability that an auditor can use. Store tamper-evident logs of what data was accessed, which tools were called, what changed, and who approved the high-risk steps. If an enterprise can’t reconstruct why an email went out or why a record changed, you’ll lose the deal during security review.
Finally: controls you can configure and prove. Domain allowlists for outbound comms. Tool allowlists by data class. Redaction rules. Retention windows. Export controls. Vague “guardrails” don’t pass procurement anymore; concrete policy settings do.
Key Takeaway
Agents don’t fail in enterprise because the model is “not smart enough.” They fail because the product doesn’t define blast radius: permissions, approvals, auditability, and rollback as built-in primitives.
Table 2: Agent readiness checklist (product capabilities that unblock enterprise rollout)
| Capability | Minimum bar | Enterprise bar | Owner |
|---|---|---|---|
| Permissions | Agent uses the active user’s access model | Scoped service identities, least privilege, time-bound scopes | Security + Platform |
| Audit trail | Store prompts and outputs | Tool-call logs, approvals, diffs, immutable ledger, export APIs | Product + Compliance |
| Cost controls | Basic rate limiting | Per-task budgets, alerts, quotas by team, showback | FinOps + Product |
| Safety & content policy | Moderation for generated text | Tool allowlists, data classification, redaction, prompt-injection defenses | Security + AI Eng |
| Rollback & recovery | Manual correction after the fact | Transactional tools, idempotency, undo flows, incident playbooks | Engineering + SRE |
Economics: price and build around resolved tasks
Token cost is trivia. Customers don’t buy tokens; they buy outcomes. The only metric that holds up in a budget meeting is cost per resolved task—total spend for a completed result, including tool calls, retrieval, retries, and human review.
This reframes model selection and architecture choices. A cheaper model that needs constant human cleanup can cost more in labor. A more expensive model that reduces rework can be the cheaper system. The same logic applies to retrieval and tool calls: many “AI costs” are really “search and integration costs.” If your agent does repeated searches, repeats calls because schemas are inconsistent, or retries because errors aren’t deterministic, you pay for it twice: in spend and latency.
- Budget tasks by risk tier (time, tool calls, and spend) and escalate deterministically.
- Force structured outputs with schemas so tool calls don’t turn into parsing chaos.
- Track human edits as first-class telemetry; rework is where ROI goes to die.
- Cap and cache tool calls the same way you would cap and cache database queries.
- Talk in resolutions, not tokens, if you want finance and procurement to take you seriously.
Pricing follows the same path. Usage-based models tied to actions or outcomes are easier to defend internally than “AI seat add-ons” that don’t map to work completed. If you can prove predictable resolution with low incident rates, you can sell expansion without turning every renewal into a debate about hype.
Rollout: staged autonomy or don’t ship
Teams break products by launching agents like they launched UI features. Don’t. Autonomy is a capability you earn through staged release: read-only, then drafts, then supervised actions, then constrained autopilot on low-risk workflows. If you skip the stages, your first incident becomes your last rollout.
Operational ownership matters as much as UX. Someone needs to own agent reliability, escalation rules, incident response, and change control. Not as theater—because without those, every failure becomes a cross-org argument about whether “AI is safe,” and adoption stalls.
- Pick three repeatable tasks users do every week. Define what “done” means in system terms.
- Build the tool layer first: stable schemas, explicit errors, deterministic permission checks, idempotent writes.
- Ship draft mode with diff-first review surfaces; gate irreversible actions behind approval.
- Stand up a task ledger that records plans, tool calls, approvals, costs, and outcomes.
- Enforce budgets (time, tool calls, spend) with clean escalation when caps are hit.
- Enable constrained autopilot only for low-risk actions, expand scope based on measured reliability.
Here’s a minimal sketch of “structured tool calling with hard budgets.” The specific SDK doesn’t matter. The discipline does: schemas, timeouts, approval gates, and ceilings the agent can’t negotiate with.
{
"task": "reconcile_invoice",
"budgets": { "maxToolCalls": 6, "maxWallTimeSec": 60, "maxCostUsd": 0.20 },
"tools": {
"search_po": { "timeoutMs": 800, "retries": 1 },
"fetch_invoice": { "timeoutMs": 800, "retries": 1 },
"post_adjustment": { "timeoutMs": 1200, "retries": 0, "requiresApproval": true }
},
"outputSchema": {
"type": "object",
"properties": {
"status": { "enum": ["matched", "mismatch", "needs_human"] },
"explanation": { "type": "string" },
"proposedAdjustment": { "type": ["number", "null"] }
},
"required": ["status", "explanation"]
}
}Next action: pick one workflow where users currently copy/paste between systems. Write down every side effect (email sent, record updated, payment issued), then design the approval gate, diff view, rollback, and audit log for each. If you can’t sketch those four pieces on one page, you’re not building an agent yet—you’re building a demo.
