AGENTIC OPS READINESS CHECKLIST (2026 EDITION)

Use this checklist to move from a demo to a production agent that is constrained, observable, and cost-bounded.

1) WORKFLOW SELECTION
- Pick one narrow workflow with a measurable outcome (example: “resolve order-status requests without a human reply where policy allows”).
- Define three metrics: outcome quality, safety/policy compliance, and efficiency (cost + latency).
- Identify the highest-risk action in the flow (money movement, deletions, sensitive data). Mark it “approval required” for v1.

2) TOOLING CONTRACTS (TREAT TOOLS LIKE APIS)
- List every system the agent can touch (Stripe, Salesforce, Jira, Zendesk, internal services).
- For each tool: define input/output schema, versioning plan, rate limits, idempotency approach, and error semantics.
- Start with read-only tools; add write tools only after logging, rollback, and idempotency are in place.

3) IDENTITY, PERMISSIONS, AND POLICY
- Create a dedicated non-human identity per agent/workflow (no shared credentials).
- Enforce least privilege per tool and per environment (dev/staging/prod).
- Add explicit policy gates for high-risk actions (refund thresholds, deletions, access to sensitive fields).
- Use short-lived credentials and allowlists for tool endpoints.

4) COST AND LATENCY BUDGETS
- Set budgets per workflow: token budget and tool-call budget. Track drift and alert on spikes.
- Add routing: small model for classify/extract; larger model only for complex reasoning.
- Cache repeated lookups (common policies, docs, known issues, stable reference data).
- Define circuit breakers: if dependencies throttle or latency spikes, degrade to draft-only mode.

5) EVALUATION BEFORE AUTONOMY
- Build a scenario suite from real, anonymized examples; include edge cases and adversarial inputs.
- Add unit tests for prompt and tool schema changes; block rollouts on regressions.
- Define required pass criteria for each action tier (low-risk actions should be near-perfect before autonomy).

6) OBSERVABILITY AND AUDIT TRAILS
- Log: prompt version, model, tool schema versions, retrieved doc IDs/hashes, tool inputs/outputs, and final action.
- Capture one trace per task that visualizes the tool-call graph.
- Build dashboards: success rate, escalation rate, policy violations, cost per task, and tail latency.

7) RELEASE AND INCIDENT PRACTICES
- Ship in stages: observe-only → draft-only → execute-with-approval → limited autonomy.
- Add a kill switch and a rollback plan for prompt/config/tool changes.
- Assign ownership: who is on-call, who triages, who approves policy updates.
- Run a regular review of top failures and fix tools/policies before expanding scope.

EXIT CRITERIA FOR “PRODUCTION READY”
- Clear SLOs with alerting.
- Least-privilege identity with enforced policy gates and audit-grade traces.
- Scenario suite running on every change.
- Hard budgets for tokens and tool calls, with circuit breakers.
- Documented escalation path and a kill switch tested in staging.