MODEL ROUTER IMPLEMENTATION CHECKLIST (v1)

Goal: Make model/provider choice cheap to change without rewriting product code.

1) Define routes (capability tiers)
- Create 4–6 internal routes (examples): FAST_CHEAP, DEFAULT, REASONING, CODE, STRICT_SAFETY, ON_PREM.
- For each route, document: intended tasks, unacceptable failures, max latency target, and whether “no answer” is preferred over a weak answer.

2) Establish a thin internal request/response contract
- Inputs: messages, optional tool definitions, required output schema (if any).
- Metadata: tenant_id, region, data sensitivity class, latency budget.
- Outputs: model/provider used, tokens/usage metadata (if available), tool calls, validation status.

3) Put routing rules in a versioned table
- Store routing config in source control.
- Require review for route changes (treat like production config).
- Support staged rollouts (by tenant, by percentage, or by environment).

4) Add policy gates before any external API call
- Data residency: block routes that can’t serve required region.
- Sensitivity: redact or force private/open-weight route for PII/IP when required.
- Tenant controls: allow/deny specific providers per customer contract.

5) Implement conservative fallbacks
- Decide per route whether to: retry same provider, try alternate provider, degrade to cheaper model, or fail closed.
- Never silently degrade for high-stakes workflows (extraction, compliance, financial ops).
- Add kill switches: provider-wide disable, model-family disable, tool disable.

6) Observability you can act on
- Log per request: route chosen, provider/model, latency, error class, validation result, fallback reason.
- Track: timeout rate, schema validation failures, tool-call failures, refusal rates.
- Make a simple dashboard that answers: “What changed?” and “Which route is failing?”

7) Build a small eval harness
- Collect a golden set of real tasks (sanitized). Include edge cases.
- Run evals on: candidate model changes, prompt/schema changes, and routing table changes.
- Gate deploy: require eval pass for any default-route modification.

8) Define incident response for model behavior
- Write a runbook: how to flip routes, disable fallbacks, and roll back.
- Decide who owns the router during incidents (one accountable operator).
- After any incident, add at least one new eval case covering the failure.

Success criterion: Switching the DEFAULT route to another provider/model is a config change plus a controlled rollout—no application code changes required.