MODEL GATE ARCHITECTURE REVIEW CHECKLIST (2026)

Goal: ensure every AI-powered workflow is traceable, testable, and swappable across models/providers without rewriting application code.

1) INVENTORY (NO GUESSING)
- List every production code path that calls an LLM (including background jobs).
- For each: what user-facing surface does it affect, and what data does it touch (public, internal, customer PII, employee data)?
- Identify direct vendor SDK usage. Flag anything not routed through a single entry point.

2) SINGLE ENTRY POINT (THE GATE)
- Define one internal API for all AI requests (even if it’s a thin wrapper at first).
- Require apps to pass: app name, environment, task type (draft/extract/rank/action), and data class.
- Block direct outbound calls to model providers via linting, code review rules, or network policy.

3) TRACEABILITY (REPRODUCIBLE OUTPUTS)
- Log: timestamp, provider, model identifier, prompt template ID + version, tool calls, tool results, and the final output.
- Decide what gets stored vs redacted for sensitive classes. Make it explicit.
- Add correlation IDs so a support ticket can point to a specific trace.

4) POLICY BUNDLES (ENFORCEMENT, NOT DOCUMENTS)
- Create named policy bundles (e.g., CUSTOMER_SUPPORT, BILLING_ACTIONS, INTERNAL_ONLY).
- Each bundle defines: logging rules, PII handling/redaction, allowed tools, and whether human approval is required.
- Enforce policies in the gate; do not rely on “engineers will remember.”

5) ROUTING + FALLBACK (OPTIONALITY)
- Define routing rules by task and constraints (latency-sensitive, long-context, multimodal, sensitive).
- Implement at least one fallback path (same provider different model, or a second provider).
- Decide degradation behavior: refuse safely, return a partial answer, or switch to a cheaper model.

6) EVALS + ROLLOUT (STOP SHIPPING BLIND)
- Build a small regression set per workflow: real prompts, expected properties, edge cases.
- Run evals on any prompt or model change before rollout.
- Add a canary: route a small slice of traffic to the new config and watch for failures.

7) TOOLING SAFETY (AGENTS NEED BRAKES)
- Maintain a strict tool allowlist per workflow.
- Add approval gates for irreversible tools (refund, delete, outbound email, deploy).
- Put hard limits on tool loops and retries; fail closed on repeated errors.

If you can’t produce: (a) a call graph of AI usage, (b) a reproducible trace for an output, and (c) an enforced policy bundle for sensitive workflows—your next milestone is not “better prompts.” It’s building the gate.