TOOL-CALLING READINESS CHECKLIST (PRODUCT + ENGINEERING)

Use this to scope and ship a tool-calling layer that can safely take actions inside your product.

1) Choose the right first workflow
- Pick a workflow with a clear start/end state (ex: “triage inbound tickets,” “prepare renewal quote drafts,” “provision user access”).
- Avoid money movement and destructive operations as your first write-capable tools.
- Define what “success” looks like in product terms (time saved, fewer steps, fewer escalations). Don’t start with model metrics.

2) Design the tool catalog as contracts
For each tool, document:
- Stable name (verb_noun): create_draft_invoice, search_customers, update_ticket_status
- Typed arguments (keep them small; prefer IDs over free text)
- Output schema (include status, IDs created/modified, and any warnings)
- Error modes (permission_denied, not_found, validation_error, conflict)
- Side effects (what data changes; what notifications fire)

3) Add execution safety by default
- Idempotency: accept an idempotency key and deduplicate server-side.
- Rate limits/quotas: per user and per workspace.
- Timeouts and retries: define what is safe to retry and what is not.
- Partial failure handling: if step 3 fails, what is the user-visible state?

4) Build approvals and preview UX
- Default to “preview” for any write: show a diff, list records affected, and summarize impact.
- Add explicit confirmation for high-impact actions (role changes, deletes, refunds).
- Provide undo/rollback when feasible; otherwise provide compensating actions.

5) Enforce permissions outside the model
- Tool execution must perform RBAC/ABAC checks server-side.
- Separate read scopes from write scopes.
- Consider field-level restrictions for sensitive attributes.

6) Make it observable (so support can debug it)
- Log: request_id, actor, tool name, args (redacted where needed), result, timestamps.
- Provide an “AI activity” view in the admin/support UI.
- Store the final structured action taken, not just assistant text.

7) Protect against prompt injection and unsafe content
- Treat retrieved content as untrusted.
- Don’t allow retrieved text to become tool instructions.
- Use allowlists: the model can only call registered tools, not arbitrary endpoints.

Release gate: You’re ready to ship a write-capable AI experience only if every tool has (a) idempotency, (b) server-side permissions, (c) preview + confirmation, and (d) an audit trail that a non-engineer can read.