AI CONTROL PLANE MVP SPEC (ONE WORKFLOW)

Goal
Define one AI-powered workflow as a reproducible, governable service. This is not a chat demo. It’s an internal product capability.

1) Workflow definition
- Workflow name:
- User-facing surface (where it appears in the product):
- Primary user job (what outcome it produces):
- Inputs (explicit fields, not “user message”):
- Output contract (JSON schema or strict format):
- Failure modes allowed (refuse, return empty, ask clarification):

2) Model policy (routing)
- Preferred model (vendor:model):
- Approved alternatives (for fallback or A/B):
- Routing rules:
 - By tenant tier? (yes/no; describe)
 - By data sensitivity? (describe)
 - By latency budget? (interactive/background)
- Timeout and retry policy:
- Fallback triggers (timeout, 5xx, format failure):

3) Prompt governance
- System prompt ID + version:
- Developer/tool instructions ID + version:
- Output-format prompt ID + version:
- Change control:
 - Who can edit drafts?
 - Who can approve release?
 - Rollback mechanism (how to revert to prior version):

4) Tools and permissions
- Tools the model may call (explicit allowlist):
- Tool schemas (versioned):
- Permission checks (what must be verified before tool calls):
- Tool error handling (what happens on tool failure):

5) Retrieval (if applicable)
- Data sources (e.g., Confluence, Google Drive, GitHub):
- Access control approach (enforced at query time; how):
- Index version (e.g., kb_rag_v3):
- Citation requirements (must cite / should cite / no citations):

6) Data handling and privacy
- What user data can be sent to the model:
- What must be redacted (PII fields list):
- Logging policy:
 - Store full prompts? (no by default)
 - Store redacted inputs? (yes/no)
 - Store retrieved snippets? (yes/no; retention period)
- Customer controls (opt-out of logging; tenant-specific policies):

7) Observability
- Trace fields to log per request:
 - trace_id
 - workflow_name + version
 - model used
 - prompt IDs + versions
 - tools invoked
 - retrieval profile + top sources
 - latency + token usage (if available)
 - output validation pass/fail
- Where traces live (vendor/tool + retention):

8) Evaluation
- Golden set: 20–50 representative cases (link/location):
- Adversarial set: 10 cases that try to break rules (prompt injection, unauthorized requests):
- Metrics tracked (qualitative allowed):
 - Format compliance pass rate
 - Grounding/citation correctness (manual rubric)
 - Refusal correctness (manual rubric)
- Release gate:
 - What must pass before deploying a prompt/model change:
 - Who signs off:

9) Release plan
- Rollout strategy (internal, % of tenants, feature flag):
- Monitoring plan (what dashboards/alerts to watch):
- Incident plan (how to disable workflow; how to force fallback model):

Definition of Done
- One internal endpoint exists for this workflow.
- Every production request produces a trace with model + prompt version.
- Output is validated against a contract (schema/format).
- A repeatable eval suite runs before shipping prompt/model changes.
- Data policy is written down and enforced (redaction + logging choices).