Your Agent Doesn't Need More Tools. It Needs a Blast Radius.
Every agent architecture review I sit in eventually arrives at the same question, and it is never about accuracy. It is: what is the worst thing this can do before a human notices? Teams that can answer in one sentence ship. Teams that cannot spend another quarter in review, and they usually blame the model.
Capability is the easy half
Getting an agent to complete a task is now largely a tooling exercise. Give a competent model a well-described set of tools, a retrieval layer that returns the right documents, and a few rounds of prompt iteration, and you will clear a demo. The demo is not the hard part and has not been for a while.
The hard part is that the same loop which makes an agent useful — observe, decide, act, repeat — is also what makes its failure modes unbounded. A traditional service fails one request at a time. An agent fails in trajectories. It retries, it reinterprets, it takes a slightly different path, and each of those steps can carry a side effect. The blast radius of a bad decision is not one action; it is every action the loop can still reach.
This is why agent projects stall at exactly the point where they become valuable. The moment you connect a tool that writes rather than reads, the risk conversation changes character, and most teams have no vocabulary prepared for it.
Per-call permissions answer the wrong question
The instinct is to reach for the access-control model you already have. The agent gets a service identity, the identity gets a role, the role grants a set of tool permissions. Every call is checked. This feels rigorous and it satisfies most checklists.
It also answers a question nobody asked. "Is this agent allowed to issue a refund?" is not the risk. The risk is the agent issuing four hundred refunds in one session because a retry loop misread a failure as a retryable error. Every one of those calls passes the permission check. Every one of them is, individually, authorized.
Stateless authorization cannot express the only constraints that actually matter for agents, because every constraint that matters is about a sequence.
The constraints you actually need sound like this: at most three refunds per session. No outbound email after reading customer PII. Cumulative spend above a threshold requires a human. None of these are expressible as a role. All of them are properties of a trace.
Design the bound before the capability
The ordering matters more than the mechanism. If you build the agent first and add constraints after the first incident, you will be retrofitting bounds onto a system whose behaviour is already load-bearing for someone. That retrofit is where projects die.
Before you connect the first write-capable tool, write down four things:
- ▹The irreversible actions. Anything that moves money, sends a message outside the organization, deletes, or changes an access grant. These get counted, capped, and logged separately from everything else.
- ▹The session budget. A hard ceiling on tool calls, tokens, and cost per session that holds regardless of what the model decides. Not a monitoring threshold — a ceiling that terminates the loop.
- ▹The taint rules. Which reads contaminate which writes. Once an agent has read untrusted input, its ability to act on privileged tools should narrow, not stay constant.
- ▹The human checkpoint. The specific, pre-agreed condition that pauses the loop and asks. If you cannot name it now, you will not be able to name it during an incident.
None of this requires a platform. All four can be enforced in a wrapper around your tool-dispatch layer in an afternoon. What they require is deciding, in advance, what the agent is not allowed to become.
Why prompt injection makes this non-negotiable
There is a category of defence that only works if the model behaves as intended, and a category that works regardless. Instructions in a system prompt telling the agent to be careful are the first kind. A counter in your dispatch layer that refuses the fourth refund is the second.
This distinction is the whole game once your agent touches untrusted input. Any content the agent reads — a support ticket, a web page, a PDF, a calendar invite — is a channel through which someone can attempt to redirect it. You cannot prompt your way out of that, because the attack lives in the same channel as the instruction. You can only bound what a successfully redirected agent is able to reach.
Treat the model as untrusted and the architecture gets simpler, not harder. You stop trying to make the agent reliably good and start making the consequences of it being wrong survivable. Those are very different engineering problems, and only one of them is tractable.
What this costs you
I would rather be honest about the trade than sell this as free. Stateful policy is genuinely harder to operate than stateless permissions. You now have per-session state that must stay consistent under retries, parallel tool calls, and process restarts. Get that state model wrong and your safety layer becomes an availability risk — agents blocked by phantom counters from a session that never properly closed.
You will also feel it in capability. A hard session budget will occasionally terminate a legitimate long task. A taint rule will occasionally block a read-then-write sequence that was perfectly fine. Those are real costs, and the temptation to loosen the bound after the third false positive is strong.
Resist it by making the bound observable rather than looser. If you can see every time a policy fired and why, you can tune it with evidence. If you cannot, you will tune it with vibes, and the direction of that drift is always toward fewer constraints.
The question to bring to your next review
Skip the accuracy numbers for the first ten minutes. Ask instead: assume the model is fully adversarial and does the worst possible thing at every step. What does it reach? If the answer takes longer than a sentence, the architecture is not finished.
The teams who can answer crisply are not the ones with the best models. They are the ones who decided what the agent was not allowed to do before they decided what it was.
Key Takeaways
- 01Bound the trajectory, not the call — every constraint that matters for agents is a property of a sequence, and roles cannot express sequences.
- 02Enumerate irreversible actions before connecting the first write-capable tool, and count them separately from reads.
- 03Enforce session budgets in the dispatch layer as hard ceilings, not as monitoring alerts that fire after the spend.
- 04Assume any content the agent reads is an injection channel; defences that depend on the model behaving are not defences.
- 05When a policy causes false positives, make it observable before you make it looser — undocumented loosening only ever drifts one way.
Sources
Enjoyed this?
Two articles a week, plus the Tuesday newsletter. Free.
One email a week, every Tuesday. No spam, unsubscribe in one click.