Agent Production Readiness Checklist (Ops-Layer Edition)

Use this to review any agent project (internal build or vendor). The goal is simple: agents must behave like disciplined services, not chatbots with keys.

1) Define the acting identity
- What principal performs actions (service account, user delegation, per-agent identity)?
- Where is identity managed (Okta, Microsoft Entra ID, Google Workspace)?
- Is there a clear human owner for each agent identity?
- Are credentials short-lived (preferred) or long-lived keys (avoid)?

2) Permission model and policy gates
- List every tool/action the agent can execute (typed actions, not “do X”).
- For each action, define required scopes and deny-by-default behavior.
- Identify “never allow” permissions (ex: blanket admin, bulk export, permission grants).
- Decide where approvals are required (money movement, production deploys, permission changes, data exports).
- Store policy as code in version control if possible.

3) Tool contracts (the most important part)
- For every tool call: schema validation, typed errors, and explicit preconditions.
- Enforce idempotency keys for side-effecting actions.
- Ensure tools return machine-parseable results (not prose).
- Block raw model output from hitting production APIs without validation.

4) Observability and audit
- Define canonical logs: tool-call events and tool-result events with trace IDs.
- Ensure logs include actor identity, timestamps, request parameters (redacted), and resulting object IDs.
- Send audit events to your existing logging/SIEM pipeline.
- Don’t treat chat transcripts as audit trails.

5) Safety, reversibility, and incident response
- For each action, document rollback or compensating steps.
- Add “break-glass” controls: immediately stop execution and revoke/rotate credentials.
- Define rate limits and spend limits (token usage, API calls, vendor quotas).
- Establish an on-call playbook: what to check first, how to disable safely, and how to recover.

6) Evaluation and change management
- Version prompts, tool schemas, and policies.
- Establish a pre-production eval suite for your specific workflows (golden tasks).
- Define what changes require review (model swap, tool additions, new permissions).
- Track regressions after vendor model updates.

Decision rule:
If you can’t answer identity + authorization + audit + reversibility clearly, the agent is not production-ready. Keep it in sandbox mode until those are resolved.