AI CHANGE-CONTROL BOARD (CCB) STARTER PACK

Purpose
Run AI behavior changes with the same discipline as payments or identity: reviewed, observable, reversible. The CCB exists to ship faster without shipping surprises.

1) CCB Membership (keep it small)
- Chair: Head of Engineering or designated Staff+ engineer with release authority
- Product owner: PM responsible for the user-facing outcome
- Security owner: security engineer or security-minded platform lead
- Data/privacy partner: whoever owns retention and access rules (can be part-time)

2) What requires CCB approval
- Model swaps (provider change, model ID change, temperature/default parameter changes)
- System or policy prompt changes
- Tool/agent permissions (new tools, expanded scopes, new side effects)
- RAG/index changes (new sources, new tenant access patterns, new indexing pipelines)
- Logging/retention changes (what is stored, for how long, who can access)

3) Required artifacts (no artifact, no ship)
A) Behavior contract (1 page)
- What the AI will do
- What it must not do (explicit “no-fly list”)
- What data it can access and how it’s segmented per tenant
- What actions it can take (if any) and user consent design

B) Evaluation bundle
- A small gold set of tasks that represent real usage
- A small set of failure tests (hallucinations, policy violations, unsafe actions)
- Known regressions list (what got worse and why it’s acceptable)

C) Observability plan
- What you trace per request: model ID, prompt version, retrieval sources, tool calls
- Where traces live and who can access
- What triggers an alert (spike in refusals, spike in tool errors, spike in user reports)

D) Rollback plan
- Exact switch or config to revert model/prompt/tooling
- Who has permission to flip it (and after-hours coverage)
- Safe degraded mode (search-only, human handoff, tool use disabled)

4) Release procedure (15–30 minutes, async-friendly)
- Pre-read posted: contract + eval results + rollout plan
- CCB review focuses on: (1) blast radius, (2) reversibility, (3) audit trail
- Decision outcomes: APPROVE / APPROVE WITH CONDITIONS / REJECT
- Conditions must be explicit (e.g., “ship to 5% cohort; tool use off by default”) 

5) Incident trigger (define this now)
Declare an AI incident if any of these occur:
- Unauthorized or unintended external action (email sent, record modified, workflow triggered)
- Exposure of sensitive data across tenants or to the wrong user
- Repeated policy violations that reach users
- A material spike in user harm signals (complaints, escalations) tied to the AI feature

First action in an incident: flip the kill switch. Then investigate. Don’t investigate while the system keeps acting.

Use this pack by pasting it into your engineering handbook and linking it from every AI project ticket template.