ICMD Agent Readiness Scorecard (2026) Use this scorecard to move from demo behavior to production behavior: predictable outcomes, controlled access, and measurable spend. A) Workflow Fit (0–10) 1) Clear Done State: Can you write an unambiguous completion condition (ticket closed with correct tag; refund issued with policy note saved; invoice posted and reconciled)? If you can’t, stop and narrow the scope. 2) Repeatability: Do you have steady volume of similar tasks? Low volume can still work if each task is high-value, but you’ll need tighter acceptance criteria. 3) Toolability: Are the required actions available through APIs (Jira, Salesforce, Stripe, Workday, ServiceNow)? If the workflow depends on UI clicks, plan for integration work or RPA. 4) Error Tolerance: What happens if the agent is wrong once? If a single failure creates regulatory exposure or large financial loss, require stricter verification and explicit approvals. B) Data & Context Readiness (0–10) 5) Source of Truth: List the systems of record. If you must reconcile many systems, plan a consolidation or a read-model step. 6) Retrieval Quality: Can you reliably fetch the right policy or doc? Run test queries and check whether the correct source appears near the top. Don’t scale until retrieval is dependable. 7) Data Hygiene: Identify PII/PHI/PCI fields and define redaction and minimum-context rules. Retrieving less reduces both risk and spend. C) Safety, Governance, and Controls (0–10) 8) Identity Model: Create a distinct agent identity per workflow. No shared keys. Least privilege per tool. 9) Tool Gateway: Put tools behind a gateway that centralizes secrets, enforces policy, and logs every call. 10) Approvals & Thresholds: Define explicit human-approval rules for sensitive writes (money movement, customer-impacting changes, high-severity incidents). 11) Auditability: Store replayable traces: prompt/version, retrieved sources, tool calls, outputs, and final actions. D) Reliability & Cost Management (0–10) 12) Golden Set: Build a curated set of real examples with expected outcomes and edge cases. 13) Verifiers: Add schema validation, deterministic checks, and for risky tasks a second-pass verifier. 14) SLOs: Define targets for completion time, escalation rate, and tool error rate. Treat misses as incidents to investigate. 15) Cost per Completion: Track total cost per verified completion (model + tools + human review). Set a budget and alert on anomalies. Scoring Guidance - High score: Ready for production write-access with a canary rollout. - Mid score: Run in sandbox or read-only mode; close gaps in retrieval, verification, or access control. - Low score: Redesign the workflow or choose a narrower bounded scope. Rollout Template Week 1: Define the contract (schemas, tools, permissions) and build a golden set. Week 2: Implement tool gateway, logging, and verifiers; run offline evals. Week 3: Shadow mode in production (no writes); measure cost, latency, and failure modes. Week 4: Canary write-access for a small slice of tasks; schedule regression runs and create an incident runbook. If you can’t measure cost per verified completion and escalation rate, you don’t have a production agent. You have a prototype.