Deterministic AI Workflow Readiness Checklist (2026)

Goal: Convert an LLM-powered “agent” into a bounded workflow you can operate (debug, audit, and control) without relying on prompts as enforcement.

1) Define the blast radius
- List every side effect the system can cause (email, Slack, ticket updates, DB writes, refunds, deletes, procurement actions).
- For each side effect, mark: reversible vs irreversible; low vs high business risk.
- Decide which actions require human approval (money movement, irreversible deletes, outbound messages to customers, bulk changes).

2) Lock down tool contracts
- Every tool must have a typed schema (JSON schema or equivalent) and strict validation.
- Reject malformed tool calls (no “best effort” parsing).
- Add idempotency keys for any action that can be retried.
- Ensure tools cannot access secrets implicitly (avoid “tool can read all env vars” patterns).

3) Put hard limits in code (not in prompts)
- Max steps per run.
- Max tool calls per run.
- Max tokens and max spend per run.
- Max runtime per run; enforce timeouts.
- Rate limits per user and per tenant.

4) Separate propose vs execute
- Model produces: structured plan, extracted fields, classification, drafted content.
- Deterministic code performs: permission checks, policy checks, execution, retries.
- Add a “dry run” mode that shows intended actions without executing.

5) Make retrieval auditable (if using RAG)
- Enforce tenant isolation (per-tenant indexes or tested metadata filters).
- Require citations for answers; if citations missing, route to fallback.
- Version documents and embeddings; define re-embed triggers on doc updates.
- Treat retrieved text as untrusted input; defend against prompt injection.

6) Add evals as a release gate
- Build a small golden set from real user queries and edge cases.
- Define success criteria per task type (correct fields, correct citation, correct action selection).
- Run evals in CI for prompt, retrieval, and tool-catalog changes.

7) Operationalize
- Emit structured logs: input, retrieved sources, plan, tool calls, tool outputs, final response.
- Provide a human-readable audit trail for every run.
- Create a runbook: what to do on loops, spend spikes, wrong-tool calls, and policy blocks.

If you can’t answer “what happened?” from logs alone, you don’t have a workflow yet—you have a demo.