Policy-First Agent Readiness Checklist (30-Day Sprint)

Goal: ship a tool-using AI feature that is controllable, auditable, and safe to roll out beyond a pilot.

Week 1 — Map blast radius
- List every tool/integration the model can call (internal APIs, email, CRM, GitHub, cloud, payments).
- Classify each tool: Read-only / Write / Irreversible / External-facing (messages to customers, web posts).
- For each tool, identify the current credential type: shared API key, per-tenant token, per-user OAuth, workload identity.
- Write down the single worst-case action per tool (be concrete: “email all contacts”, “delete repository”, “export invoices”).

Week 2 — Fix identity and scopes
- Ensure end-user identity is preserved end-to-end (every run ties back to a real user and tenant).
- Eliminate shared “god tokens” for high-risk tools; move to per-user or per-tenant scoped credentials.
- Define least-privilege scopes for each tool (what read/write operations are allowed, and to which resources).
- Add environment separation: sandbox/staging/prod. No agent testing against prod data by default.

Week 3 — Build the tool gateway
- Route all tool calls through one gateway layer.
- Deny by default: tools are off unless explicitly allowed per tenant/workspace.
- Add deterministic validation for tool arguments (types, limits, allowed domains, allowed resource IDs).
- Add rate limits and quotas per user/tenant.
- Implement “human approval required” for irreversible or external-facing actions.

Week 4 — Make it operable
- Add run traces: user, timestamp, model, tool calls, tool outputs, final result.
- Add redaction rules so logs are useful without leaking secrets.
- Add operator UI controls: tool allowlists, kill switches, approval queue, retention settings.
- Add regression tests for “never do this”: cross-tenant retrieval, external email domains, exporting sensitive datasets.

Launch gate (do not skip)
- A new tenant starts with zero write tools enabled.
- You can answer: “Which user triggered this tool call?” for any run.
- You can disable a tool instantly without redeploying.
- You can show an auditor a clear, human-readable trace of a run.

One question to keep on the wall: If the model goes off the rails, what exactly stops it—code, or a prompt?