Production Agent Readiness Checklist (2026 Edition)

Use this before an agent is allowed to write to core systems (CRM, payments, ticketing, source control, IT).

1) Scope & success metrics
- Pick one workflow the agent owns (one system of record plus a small number of downstream actions).
- Define acceptance criteria as measurable targets (quality, latency, escalation rate).
- Set stop conditions (wall-clock timeout, tool-call cap, retry cap).
- Define escalation rules (what routes to a human; what requires manager approval).

2) Tool contracts (propose vs. execute)
- Put every tool behind a strict JSON/schema contract with required fields and constrained values.
- Validate model outputs before executing any tool.
- Require idempotency keys for every write action (refunds, tickets, PRs, CRM updates).
- Separate propose from execute: the model suggests; policy code approves/denies.

3) Identity, permissions, and secrets
- Create a dedicated agent identity (OAuth client/service account) per agent.
- Enforce least privilege; document why each permission exists.
- Keep static secrets out of prompts, logs, and vector stores; rotate credentials on a schedule.
- Rate-limit high-impact tools and add circuit breakers.

4) Data & memory hygiene
- Separate memory types: session state, long-term knowledge, episodic logs.
- Set retention rules per type; version what must be reproducible.
- Detect and redact PII before storage and before sending content to model APIs.
- Ensure retrieval sources are traceable (which document/section supported the decision).

5) Evaluation & release process
- Build a regression suite of real tasks and keep it under version control.
- Track: task success rate, tool-call correctness, escalation rate, grounded-answer accuracy.
- Use canary releases with an instant rollback switch.
- Define a no-ship threshold (if the eval suite degrades, stop the release).

6) Observability & incident response
- Emit structured traces per run: prompts, retrieval hits, tool calls, tool outputs, validations, approvals.
- Dashboard latency, failure rate, cost per run, and top escalation reasons.
- Alert on abnormal behavior: duplicates, retry spikes, spend anomalies.
- Write an incident playbook: disable switch, rollback steps, and comms plan.

7) Unit economics & budget controls
- Set unit cost targets per completed task.
- Use model routing (cheap router + stronger solver) with explicit escalation criteria.
- Enforce spend caps per day/week and per tenant; fail safe to human handoff.
- Track wasted spend: tokens on failed runs, retries, and rejected tool calls.

Launch gate
You’re ready when: (a) every write tool is protected by schemas and policy gates, (b) success and cost per completed task are measurable, (c) any action can be audited end-to-end, and (d) the agent can be disabled quickly.