An agentic system was being handed real work, the kind where an action changes something outside the conversation. Two problems stood between it and being trusted with that work, and both were silent. Each one produced a result that looked like success. Neither is new. They are old distributed-systems problems arriving in new clothing, and they decide whether an agent can be given anything that matters.
An agent is a loop that calls tools, and every tool call is a message sent across a network.
Strip away the vocabulary and an agent is a loop. A model reads the task and the history so far, picks an action, and calls a tool to perform it. The tool might search a database, send a message, update a record or charge an account. The result comes back, the model reads it, and picks the next action. The loop runs until the model decides it is done or something outside it calls a halt.
Each tool call is a request to another system, and requests across a network have a property that is easy to forget. When one times out, the caller does not know what happened. The request may have been lost on the way, in which case nothing was done. Or the work may have been done and the reply lost on the way back. From the caller's side the two cases look identical. Distributed systems have names for the choices this leaves. Never retry, and some work is lost. Retry until a reply arrives, which is called at-least-once delivery, and some work may be done twice. The standard way to get the effect of exactly once is to retry and make the operation idempotent: each request carries a unique key, the receiving side records which keys it has already handled, and a repeat simply returns the earlier result.
Stopping has a similar trap. A cap on iterations is a circuit breaker, a safety device that cuts power to a runaway loop. It carries no information about whether the task is complete. Completion needs its own definition, written per task, as something that can be checked in the world: the record exists, the message was delivered, the report has every section. With that test in place a run can end in three distinguishable ways. It completed, it stopped on budget, or it escalated because it reached a state it could not verify.
- Agent calls a side-effecting toolthe request carries a unique key
- The tool performs the actionthe change in the world has happened
- The reply is lostthe agent sees only a timeout
- Agent retries with the same keyit cannot know the first call landed
- Tool recognises the keyreturns the first result, does nothing new
The first problem is the iteration cap. A cap is a circuit breaker. It stops a runaway loop, and that is all it does. A system that halts at step twenty returns the same shape of result whether it completed the task or ran out of budget, and nothing downstream can tell which. The second problem is the timeout. When a tool call times out, nobody knows whether the first attempt went through. A naive retry is then a second execution, and on any action that is not idempotent, that means the side effect happens twice.
Agent failures look like success, which is why they get through testing.
Most agent testing checks whether the agent produced an output and whether the output looks right. Both failures here pass that test. An agent that ran out of iterations still produces a final message. An agent that retried a timed-out action still reports a single, successful result. The defect sits in the world outside the transcript, where a task is half complete or an action happened twice.
The second reason these failures survive is that they are rare in testing and common in production. Timeouts are infrequent on a developer's machine and routine under real load. Long tasks that exhaust their budget appear only when the inputs are larger and messier than the examples. By the time the failures show up they are already in customer data.
The reflex, and the fix.
Cap iterations and retry on timeout
Two lines of code, and it stops the obvious runaway.
It treats stopping as finishing, and it turns every uncertain step into a possible duplicate execution.
Explicit completion conditions and idempotency keys
Defining what done means for each task, and making every side-effecting step idempotent so a retry is safe.
A system that knows whether it finished, and one that can retry without doing the work twice.
Treat the agent as a distributed system, because that is what it is, and use the discipline that field already has.
Done is a test, written per task
Each task carries an explicit check of success. Completing and running out of budget become different states, with different handling downstream, where before they produced identical output.
A key on every side-effecting step
Each action carries an idempotency key. A retry after an uncertain timeout is recognised as the same operation and is never performed a second time. This is the standard at-least-once discipline, and agent systems routinely ship without it.
Stop and hand off on unverified state
A step whose result cannot be verified halts the chain and escalates. Unverified state is never passed along to every step that comes after it.
What each choice cost, and what it bought.
AWriting a definition of done for each task
The cheap version is a single iteration cap for everything. Writing an explicit success test per task is slower, and it forces awkward conversations about tasks nobody had defined precisely. What it buys is a run that can report one of three honest outcomes: completed, stopped on budget, or escalated. Downstream systems can then treat a partial run as partial. The cap still exists, but it goes back to being what it is, a guard against runaway loops, and it is no longer mistaken for a finish line.
BPutting the idempotency key on the action itself
It is tempting to handle retries once, in the orchestration layer, and leave the tools alone. That fails because only the action knows what makes two requests the same request. So each side-effecting step was given its own key and its own record of what it has already done. That is more work per tool, and every new tool has to follow the pattern. It buys retries that are always safe, which means the agent can be allowed to retry at all.
CEscalating on any step that cannot be verified
A chain that continues past an unverifiable result usually finishes, which is exactly the problem. Each later step inherits the doubt and adds its own. Halting and handing the case to a person costs throughput and looks less impressive in a demo. It buys a system whose completed runs can actually be trusted, because nothing in them rests on a result nobody confirmed.
DDeclining to automate untestable tasks
Some tasks have no checkable definition of done. Automating them anyway produces an agent that always stops on budget and always looks finished. Leaving them with people narrows the scope of the system. It also keeps every automated task inside the set where the system can tell success from stopping.
A single headline number hides where a system fails. This work was scored on the dimensions that actually decide whether it holds in production, measured on real, held-out cases rather than the demo path.
A system that reports success when it has merely run out of iterations, and that now and then performs the same side effect twice because a timeout was read as a failure.
An agent that knows whether it finished, retries safely because every side-effecting step is idempotent, and escalates when it cannot verify the state it is standing on.
What it owns, and what it hands to a person.
Idempotency has to be designed into each action. It cannot be bolted on generically at the orchestration layer, because only the action knows what counts as the same request. Tasks whose success cannot be tested cannot be given a completion condition, and those tasks should not be automated.
Almost everything hard about agent reliability is already solved in distributed systems: at-least-once delivery, idempotency, circuit breaking and explicit state machines. Most of the work is recognising which classical problem is in front of you.
An iteration cap tells you an agent stopped, never that it finished, so every automated task needs its own checkable definition of done.
A timeout means the outcome is unknown, and retrying without an idempotency key turns uncertainty into a duplicated action.
Idempotency has to live with each action, because only the action knows what makes two requests the same.
A chain should stop and escalate on any result it cannot verify, because every later step inherits the doubt.
Most hard problems in agent reliability already have names and solutions in distributed systems, and recognising which one you face is most of the work.