Agent Runtime Readiness Checklist (2026)

Use this before you ship any tool-using AI feature. It’s designed to force hard decisions on scope, permissions, observability, and failure modes.

1) Define bounded agency (product constraints)
- Write the single sentence contract: “This agent can do X, cannot do Y.”
- List allowed tools by name. Anything not listed is forbidden.
- Set a tool-call budget per task (a hard stop).
- Decide your default mode: read-only, draft-only, or write-capable.
- For every write action, define the user-visible confirmation pattern (auto, confirm, admin approve).

2) Tool contract (engineering constraints)
- Every write tool requires an idempotency key.
- Every tool returns structured errors: retryable vs fatal vs permission.
- Tools must have explicit scopes: resource-level constraints, not just “can call API.”
- Tools must support timeouts and cancellation.
- Define redaction rules for inputs/outputs before logging.

3) State & control flow (runtime design)
- Use an explicit state machine/graph for execution (not an unbounded loop).
- Store durable task state outside the model (DB or durable store).
- Implement safe retries with backoff and max attempts.
- Add a deterministic escape hatch: if the agent can’t progress, it stops and asks.
- Add invariants: “never email external domains,” “never delete records,” “never change billing.”

4) Observability (must-have to operate)
- Trace every tool call: args (redacted), output, latency, error type.
- Log policy decisions: why an action was allowed/blocked.
- Provide a replay mechanism for incidents (same inputs, same tool responses when possible).
- Build an operator view: task timeline, steps, and current state.

5) Evaluation (tests that map to incidents)
- Permission tests: downgraded roles, blocked tools, attempted bypass.
- Failure injection: timeouts, rate limits, 500s, partial responses.
- Ambiguity tests: missing identifiers, conflicting user instructions.
- Drift tests: mutate data mid-run; ensure agent re-validates.
- Regression suite: keep a fixed scenario pack and run it on every change.

6) Launch controls (reduce blast radius)
- Start with a narrow cohort or internal users.
- Add rate limits and per-user quotas.
- Add a kill switch for tool execution (global and per tool).
- Create a runbook: common failure modes, escalation path, how to disable writes.

If you can’t check most of the above, you’re not ready for an agent. You’re ready for a draft generator with a button.