The fastest way to tell whether a company is shipping “AI agents” or shipping a demo: ask where the agent’s identity lives, how its spend is capped, and how you can reconstruct every side effect it caused.
If the answer is “it’s just an API key in an.env file,” you don’t have an agent. You have an unbounded script with a personality layer.
2026 is the year this stops being cute. Agents are now glued to real systems: GitHub repos, Slack workspaces, Stripe accounts, CRMs, Kubernetes clusters. The hard part isn’t the LLM. It’s identity, authorization, and forensics in a world where the “user” is a piece of software that speaks natural language and calls tools.
Agents don’t fail because they can’t think. They fail because you didn’t give them a safe way to act.
What changed: agents now touch money, code, and production systems
Three public trends collided and made agent governance unavoidable.
First: models got good enough that teams started wiring them to toolchains by default. OpenAI’s Assistants-style tool use, Anthropic’s “computer use” direction, and the broader ecosystem around LangChain and LlamaIndex normalized the idea that the model should browse, click, run commands, open PRs, and update tickets.
Second: SaaS vendors started shipping agent-shaped product surfaces. Microsoft has Copilot across Microsoft 365 and GitHub Copilot for code. Salesforce pushed Agentforce for CRM workflows. Atlassian has been embedding AI across Jira and Confluence. Whether you love these products or not, they moved “agentic work” from labs into procurement.
Third: regulators and security teams stopped accepting “it’s just an LLM” as an explanation. The EU AI Act is real law. NIST’s AI Risk Management Framework is not law, but it’s what a lot of enterprise risk people point to because it’s concrete. And every security leader has lived through at least one incident where an over-permissioned integration caused damage at machine speed.
The contrarian take: stop calling them “agents” — start treating them like service accounts
The industry keeps arguing about “autonomy levels” and “reasoning.” That’s mostly a distraction. In production, an agent is an identity that can:
Authenticate to systems (often many of them).
Receive inputs (tickets, emails, prompts, events).
Call tools (APIs, CLIs, browser actions).
Create side effects (write code, send emails, move money, delete data).
Leave an audit trail you can explain to humans.
Once you accept that, the right mental model isn’t “new employee.” It’s “service account with a natural-language interface.” That’s good news: the world already knows how to control service accounts. Bad news: most agent pilots ignore the basics that security teams spent a decade enforcing for bots.
Key Takeaway
If you can’t rotate the agent’s credentials, scope its permissions, cap its spend, and replay its actions from logs, you’re not deploying an agent. You’re deploying a liability.
Identity: “who is the agent?” needs to be answerable in every system
Most agent setups still run on a single shared API key and a pile of OAuth tokens in a database. That’s how you end up with the worst sentence in operations: “We can’t tell which actions were human and which were the agent.”
In 2026, the only sane posture is that every agent has its own identity, and every tool call is attributed to that identity.
Where identity should live: your IdP, not your prompt
If your company uses an identity provider like Okta, Microsoft Entra ID (Azure AD), or Google Cloud Identity, agent identities should be first-class there. Treat them like non-human principals with:
Unique names and owners.
Scoped group membership.
Strong authentication (short-lived tokens, not static secrets).
Lifecycle management (creation, rotation, decommissioning).
In cloud infrastructure, that often means IAM roles and workload identity rather than long-lived keys. On Kubernetes, it means service accounts and tight RBAC. On GitHub, it means GitHub Apps (or fine-grained tokens) rather than a developer’s personal token taped to the agent server.
Stop sharing “the agent” across workflows
One agent that “does everything” is convenient for a demo and awful for governance. You want multiple narrow identities: one agent that triages support tickets, another that proposes code changes, another that manages cloud cost reports. Least privilege only works if you actually separate privileges.
Authorization: tool permissions are your real model alignment
People talk about “alignment” as if it’s a property of the model. In production, alignment is a property of permissions. The agent can only do what the tool layer allows.
This is where teams get sloppy: they over-grant access to avoid breaking flows, and they rely on the model to “behave.” That’s backwards. If the agent can delete customer data, it eventually will—because a tool call got constructed wrong, because an input was ambiguous, because a ticket template changed, because someone pasted malicious instructions into a document the agent reads.
Prefer allowlists over natural-language “policies”
Write policies in code, not English. Your agent runtime should enforce an allowlist like:
Which tools exist.
Which endpoints per tool.
Which parameters are permitted (and which are forbidden).
Which resources can be touched (repo allowlist, Slack channel allowlist, Jira project allowlist).
Natural-language policies are fine as documentation for humans. They are not a control surface.
Use “two-person rules” where the blast radius is existential
Some actions should require explicit approval from a human identity: merging to a protected branch, issuing refunds, rotating production secrets, deleting data, changing IAM policies. You already have patterns for this: GitHub branch protection, required reviews, and CI checks. Reuse them.
Table 1: Common agent runtimes/frameworks and what they’re actually good for
| Tooling | What it is | Best fit | Operational caveat |
|---|---|---|---|
| LangChain | Open-source framework for chaining LLM calls and tools | Prototyping tool use; integrating many connectors | You must design auth, logging, and guardrails yourself |
| LlamaIndex | Open-source data/RAG framework with agent patterns | Knowledge-heavy agents grounded in internal docs | Grounding isn’t governance; tool permissions still decide outcomes |
| OpenAI API (Responses/Assistants-style tool calling) | Hosted model + structured tool calling | Fast path to reliable tool invocation patterns | You still own identity mapping and auditing across your systems |
| Anthropic API (tool use) | Hosted model + tool calling; strong emphasis on safe behavior | Workflows where refusal and caution are desirable defaults | Safety posture doesn’t replace strict allowlists and approvals |
| Microsoft Copilot (M365/GitHub) | Productized assistants inside Microsoft ecosystem | Enterprises standardized on Microsoft identity and controls | Cross-system actions still need careful connector scoping |
Budgets: cap cost and cap blast radius (they’re the same problem)
Everyone thinks “budgeting” means API spend. That’s the small part. The real budget is operational: how many external calls per hour, how many writes per day, how many tickets it can close without review, how many PRs it can open, how many emails it can send.
Unbounded agents fail in two ways: they run up bills (model calls, tool calls, SaaS actions), and they create cascading side effects. The fix is the same: quotas.
Budgets you can enforce in code
Token and request caps per agent identity, per time window.
Tool-call caps: max actions per run; max actions per day.
Write caps: read is cheap; writes are expensive. Separate the two.
Scope caps: limit which repos, projects, or customers an agent can touch.
Timeouts and circuit breakers for external dependencies.
A concrete pattern: “plan, then execute” with a spend envelope
Don’t let the agent stream actions until it hits a goal. Make it produce a structured plan first, then execute under a pre-approved envelope. If the plan changes, it re-requests approval. This can be fully automated for low-risk scopes, and human-gated for high-risk ones.
# Example: enforce per-run quotas in an agent executor (pseudocode)
MAX_TOOL_CALLS=20
MAX_WRITES=5
state.tool_calls=0
state.writes=0
function call_tool(name, args):
state.tool_calls += 1
if state.tool_calls > MAX_TOOL_CALLS:
raise QuotaExceeded("tool calls")
result = tools[name](args)
if result.side_effect == "WRITE":
state.writes += 1
if state.writes > MAX_WRITES:
raise QuotaExceeded("writes")
log_event(agent_id, name, args, result)
return result
Audit trails: if you can’t replay it, you can’t run it
An agent that can’t be audited is a non-starter in any serious environment. And “we store chat logs” doesn’t count.
You need an event trail that ties together:
The input that triggered the run (ticket ID, webhook payload, user request).
The model configuration (provider, model name, system instructions, tool schema version).
Every tool call (arguments, timestamps, responses, errors).
Every side effect in the target system (PR URL, Slack message link, Jira transition, Stripe object ID).
The human approvals (who approved, what they saw, when they approved).
That’s the difference between “the agent did something weird” and “we can show exactly what happened and roll it back.”
Observability you already have — wire agents into it
Most teams already run centralized logs and traces. Put the agent runtime inside that world instead of building a separate “AI dashboard” that only shows prompts.
Use OpenTelemetry for traces if you’re already standardized there. Use structured logging so you can query by agent_id, run_id, and tool_name. Store tool I/O in a way that’s searchable and access-controlled; these payloads often contain customer data and secrets.
Table 2: A production-readiness checklist for agent deployments (identity, budgets, auditability)
| Control | Minimum bar | What to log | Owner |
|---|---|---|---|
| Agent identity | Dedicated non-human principal (IdP/IAM), not shared keys | agent_id, credential type, token TTL, key rotation events | Security + Platform |
| Tool allowlists | Explicit tools/endpoints/resources allowed; deny by default | tool schema version, endpoint, resource IDs, denied calls | Platform |
| Write gating | Protected actions require approvals or policy checks | approval_id, approver identity, diff/preview, final action IDs | Product + Security |
| Budgets & quotas | Caps on model calls, tool calls, and writes per window | quota config, quota hits, run termination reasons | Platform + FinOps |
| Forensic replay | Reconstruct full run from input → prompts → tool calls → side effects | run_id, input refs, model params, tool I/O hashes, external object links | SRE |
The founder/CTO trap: “we’ll harden it after product-market fit”
That logic works for UI polish. It fails for agents because the first real customers you win are also the first customers who will demand audits, SOC 2 narratives, access controls, and incident response clarity.
If you’re selling an agent that operates inside a customer’s environment, the security story is the product. Your competitors will claim “enterprise-ready” and ship a checklist. If you can’t answer basic questions about identity, permissioning, and logs, you’ll lose deals to companies with worse models and better controls.
Why this is a moat for small teams
Big companies often ship agents bolted onto existing permission systems with messy inheritance and legacy admin surfaces. Smaller teams can win by being opinionated: strict scoping, explicit approvals, clean audit exports, predictable failure modes.
Make it easy for a security engineer to say yes. That’s not marketing; it’s roadmap selection.
What to do next week: ship an “agent control plane” before you ship more prompts
If you’re already running an agent in production, don’t start by rewriting prompts. Start by making the agent governable.
Create one dedicated identity per agent in your IdP/IAM and remove shared credentials.
Write an allowlist for tools, endpoints, and resources. Default deny.
Add quotas for tool calls and writes, not just model tokens.
Implement approvals for the actions that would create an incident if wrong.
Centralize audit logs and make “replay a run” a first-class on-call skill.
Prediction worth sitting with: by late 2026, “agent platforms” won’t be differentiated by which model they call. They’ll be differentiated by whether a security team can reason about them like any other system: identities, policies, and evidence. If your agent can’t produce receipts, it won’t get keys to the building.
Your next action is simple: pick one agent you’re proud of, and try to answer this without hand-waving — exactly what could it do if a malicious instruction landed in a Jira ticket it reads? If you can’t bound the answer, you know where to start.
