MODEL CONTEXT PIPELINE SPEC (ONE-PAGE TEMPLATE)

Use this to spec ONE high-stakes LLM workflow (support reply, refund decisioning, incident assistant, sales ops, code change assistant). Keep it tight; if it takes more than a page, you’re hiding ambiguity.

1) Workflow Definition
- Name:
- User persona(s):
- Primary goal (what must be true when it succeeds):
- Failure cost (qualitative): low / medium / high

2) Allowed Outputs
- Allowed output types: (e.g., draft reply, structured JSON, tool calls)
- Must include: (e.g., citations, confidence flags, links to records)
- Forbidden: (e.g., policy interpretation, legal advice, writing to DB)

3) Context Sources (list ONLY what the model may see)
For each source, fill:
- Source system: (e.g., Postgres orders DB, Zendesk, Confluence)
- Data class: authoritative state / policy / history / knowledge
- Access control basis: (role, tenant_id, org_id, doc ACL)
- Freshness requirement: (must be live query? acceptable staleness?)
- Provenance fields required in logs: (URL/path, doc_id, updated_at, owner)

4) Retrieval + Assembly Rules
- Retrieval modes: lexical, vector, hybrid
- Filters enforced BEFORE retrieval: (tenant, role, tags, collections)
- Token budget per class: (state vs policy vs docs)
- “Must-call-tool” intents: (e.g., order status, account entitlements)
- Citation rule: which outputs require citations and what format

5) Tool Policy (Actions)
For each tool:
- Tool name:
- Allowed intents:
- Required parameters and constraints:
- Rate limits / timeouts:
- Human-in-the-loop required? (yes/no; trigger conditions)
- Audit log fields: (inputs, outputs, request_id, user_id)

6) Security Controls
- Untrusted content sources (explicitly list):
- Sanitization steps: (strip instruction-like text, down-rank certain patterns)
- Prompt injection handling: (treat retrieved text as data, never instructions)
- Data exfiltration guardrails: (PII redaction, denylist topics, refusal rules)

7) Evaluation Checklist (ship-blockers)
- Permission retrieval test: synthetic users + fixtures pass
- Retrieval quality set: curated query→source pairs pass
- Citation coverage: required claims have source IDs
- Tool-plan policy test: only allowed tools chosen for given intents
- Freshness guardrail: stateful intents always query live state

8) Observability (what you log)
- Context package log: retrieved source IDs + timestamps + filters applied
- Tool trace log: tool calls, parameters, outputs, errors
- User feedback capture: thumbs up/down + reason + linked trace ID
- Red-team scenarios: top 10 failure prompts and expected behavior

Sign-off
- Product owner:
- Engineering owner:
- Security/Compliance reviewer (if applicable):
- Date: