Production LLM System Readiness Checklist (Evals + Controls)

Use this as a gate before you call an LLM feature “production.” If you can’t check an item, treat it as engineering debt with a clear owner.

1) Define the job, not the model
- Name one workflow you own end-to-end (e.g., “draft a support reply in Zendesk that follows policy X”).
- Write the explicit constraints: allowed sources, forbidden data, required format, and what “done” means.
- Decide what the system may do automatically vs what requires human approval.

2) Instrumentation you must have
- Per-request trace ID across: prompt template, retrieved docs, tool calls, and final output.
- Log model name/version (or provider + model string), latency, and token counts.
- Store retrieval artifacts: query, doc IDs, chunk IDs, rank order.
- Store tool-call ledger: tool name, arguments, auth context, response, and error states.

3) Evals: minimum viable suite
- Build a small golden set (start with 25–100 cases) that reflects real tasks.
- Add an adversarial set: prompt injection attempts, tricky inputs, and policy-edge cases.
- Create regression gates: prompt changes, retriever changes, embedding changes, model/provider changes.
- Prefer deterministic checks where possible: required citations, required fields, prohibited content, JSON schema validity.

4) Retrieval quality controls (if you use RAG)
- Define “sources of truth” and block everything else by default.
- Use metadata filters (customer, product, region, time) instead of relying on embeddings alone.
- Track staleness: how you re-index, how you remove outdated docs, and how you handle duplicates.
- Measure retrieval failures separately from generation failures.

5) Action safety (if the model can change state)
- Require explicit tool schemas and validate arguments before execution.
- Add allow/deny policies per tool and per user role.
- Put approvals on high-risk actions (refunds, deletes, permission changes).
- Build idempotency and rollback paths where possible.

6) Portability and vendor risk
- Wrap all model calls behind an internal interface (prompt IDs + structured output contracts).
- Keep eval datasets and traces provider-agnostic so you can compare models.
- Maintain at least one fallback route (smaller model, alternative provider, or degraded mode).

7) Launch gates
- Red-team your top failure modes and add tests for each one.
- Add monitoring for drift: rising refusal rates, rising tool errors, rising retrieval mismatch.
- Define an incident playbook: what you disable first, who owns comms, how you audit impact.

If you can check everything above for one workflow, you have something rare: an LLM system that behaves like software, not a demo.