Governed Answers Build Sheet (Graph + RAG)

Use this as a build checklist for turning a “chat with docs” feature into a system that can be audited, scoped, and maintained.

1) Pick the workflow (one only)
- Name the workflow: __________________________
- What decision does the answer influence? __________________________
- What is the “blast radius” if the answer is wrong? (Ops, legal, revenue, security): __________________________

2) Define canonical entities (system-of-record)
List the smallest set of entities the workflow depends on:
- Entity: __________ System of record: __________ Unique ID field: __________
- Entity: __________ System of record: __________ Unique ID field: __________
Examples: Customer, Contract, Plan/Tier, Region, Policy, Product/SKU, Ticket, Incident.

3) Model claims (don’t store “facts”)
For each claim type, define:
- Claim statement template (e.g., “Refund window is {N} days”): __________________
- Required provenance: source_doc_id, span/section, author/owner, effective_date
- Status lifecycle: draft → approved → superseded → retired

4) Encode precedence rules (how contradictions resolve)
Choose explicit rules and write them down:
- Supersession: newer policy supersedes older policy (yes/no): ____
- Contract overrides policy (yes/no): ____
- Region-specific policy overrides global (yes/no): ____
- Authority ranking (which team can publish “final”): ____

5) Permissions and scope (before retrieval)
- User context attributes you will enforce: role, workspace, region, customer, clearance
- Document/entity permission tags: __________________________
- Rule: retrieval MUST only search within allowed tags (no post-generation filtering)

6) Retrieval plan (hybrid, not vibes)
- Keyword search engine (Elasticsearch/OpenSearch) index fields: title, headings, tags, effective_date
- Vector index purpose: candidate passage retrieval only
- Required metadata filters: permission_tags, region, effective_date, status=approved

7) Audit log (replayable answers)
Store these fields for every answer:
- Query + user context snapshot
- Graph ruleset version (or migration version)
- Retrieved doc IDs + exact spans/offsets
- Model name/version + prompt template version
- Final answer + citations
This is your debugging and compliance substrate.

8) Change management
- What constitutes a “high impact” change? (policy update, contract template update, pricing update)
- On change event, create an “affected answers” review queue tied to entities/claims
- Define an owner who approves re-publication of answers for that workflow

9) Acceptance tests (minimum set)
Write 8–12 test questions with expected citations and scope:
- 3 normal cases
- 3 edge cases (conflicts, missing data, ambiguous entity names)
- 2 permission cases (should refuse or provide limited answer)
- 2 freshness cases (new doc supersedes old)

If you can’t fill this sheet for one workflow, you don’t have a knowledge problem—you have an ownership problem.