MODEL ADAPTER LAUNCH CHECKLIST (WORKFLOW-FIRST)

Goal: ship a durable “adapter” layer that survives model churn and passes enterprise scrutiny.

1) Pick the wedge workflow (do this before code)
- Name the workflow in 6 words or less (e.g., “Support reply drafting with citations”).
- Write ONE failure sentence: “The system fails if ____.” Examples:
 - “It includes PII in outputs.”
 - “It cites a source that wasn’t retrieved.”
 - “It changes CRM fields without approval.”
- Identify the system boundary: what data can cross it, and what must never leave.

2) Define a stable internal schema
- Messages: roles, timestamps, conversation IDs.
- Tools: tool name, input JSON schema, output JSON schema.
- Retrieval: document IDs, snippets, source URLs/paths, permission context.
- Output contract: a JSON schema you validate (not “best effort”).

3) Policy layer (ship early)
- Redaction rules: what gets removed or masked before vendor calls.
- Allow/deny tools per workflow and per user role.
- Data retention: what you store (prompts, outputs, traces), for how long, and who can access it.
- “No-train/no-log” settings per provider where available, documented for buyers.

4) Observability (make it operable)
- Emit request IDs across the stack.
- Capture traces for: prompt construction, retrieved docs, tool calls, model responses, post-processing.
- Store versions: prompt version, tool schema version, model ID, policy version.

5) Evals (the upgrade gate)
- Build a golden set: 20–50 real scenarios from the workflow.
- Define pass/fail checks tied to the failure sentence:
 - schema validation
 - forbidden content checks (PII, secrets)
 - citation integrity (must reference retrieved docs)
- Run evals before any model swap, prompt change, or tool schema change.

6) Routing (only after evals exist)
- Define routing rules that are explainable:
 - by modality (text vs. vision)
 - by risk tier (sensitive vs. non-sensitive)
 - by latency needs (sync vs. async)
- Keep a manual override and a rollback switch.

7) Security & procurement readiness
- Create a one-page “AI System Facts” doc:
 - data flows (what leaves, what stays)
 - logging/retention policy
 - supported model providers and how switching works
 - audit artifacts available (traces, approvals, version history)

Exit criteria (you’re ready to sell)
- You can reproduce a past output: same input + same versions = same trace path.
- You can swap model providers without changing the application’s core logic.
- You can show a buyer the failure definition, the eval gate, and the rollback plan.