Verifiable AI Ship Checklist (v1)

Use this checklist before you call an LLM feature “production-ready.” The goal isn’t perfection. It’s being able to prove what happened and prevent the worst failures.

1) Define the blast radius
- List every external system the AI can touch (email, Slack, CRM, billing, GitHub, cloud infra).
- For each, write down the worst plausible failure (data leak, unauthorized change, irreversible action).
- Decide which actions must be “read-only” until you have stronger controls.

2) Make tool use explicit and enforceable
- Represent each tool as a typed function with a strict schema (no free-form parameters).
- Put authorization outside the model. The model proposes; a policy engine decides.
- Start deny-by-default. Create an allowlist per role/tenant and per environment (dev/stage/prod).
- Add rate limits and idempotency keys for any write action.

3) Build a chain of custody for answers
- Store: input text, user/tenant, model name/version, settings, and prompt template version.
- If you do retrieval, store exactly which documents/chunks were used, with immutable IDs.
- If the system returns “facts,” require citations or label the output as ungrounded.

4) Observability you can replay
- Attach a trace_id to every request and propagate it through retrieval and tool calls.
- Log tool call requests/responses (with secrets redacted) and policy decisions.
- Confirm you can reconstruct a single bad outcome end-to-end from logs alone.

5) Evaluation gates (don’t ship without tests)
- Create a small, curated set of test prompts that represent real user requests.
- Add adversarial cases: prompt injection, requests for secrets, and unsafe actions.
- Run evaluations in CI on every change to prompts, tools, and retrieval settings.
- Record failures as issues with a reproducible input and expected behavior.

6) Change management and rollback
- Version prompts and policies like code.
- Use staged rollouts (internal → pilot customers → broad release).
- Have a kill switch: disable tool calling and fall back to read-only mode.

7) Customer-facing evidence
- Provide an “activity log” view: what sources were used and what actions were taken.
- Make exports possible for audits (structured logs, not screenshots).

If you can’t complete sections 2–4 for a workflow with real side effects, keep it in read-only mode. That’s not being cautious. That’s being shippable.