Why Your AI Agent Won't Stop (and Why Capping the Loop Doesn't Fix It)
An AI agent can run green for 264 hours and bill 47,000 dollars with every single call bounded, because the iteration cap is a fuel gauge, not a progress bar. It measures how much you have spent, not how much work is left, so 'I gave up' and 'I am done' come back identical. Stopping reliably requires a number that measures the task itself, computed without asking the model.
Why Your AI Agent Won't Stop (and Why Capping the Loop Doesn't Fix It)
The short answer. An agent can run green for 264 hours and bill 47,000 dollars with every call individually bounded, because the iteration cap is a fuel gauge, not a progress bar. It measures how much you have spent, not how much work remains, so a run that gave up and a run that finished come back looking identical. Stopping reliably needs a number that measures the task, computed in code without asking the model, plus a real third state for "gave up."
Here is a real system, numbers changed enough that you cannot tell whose it is. Status green, every monitor says fine, no errors, no stack traces, no alerts, and it ran for 264 hours straight, 11 days, and billed 47,000 dollars. Week one was 127 dollars, below the line where anyone checks an invoice. Week two was 1,340. Week three, six thousand. Week four, eighteen thousand. One bounded run of the same agent costs about 100 dollars. And the team was not careless. They had already capped the loop, set max durations, and called it a day. Every call in the system was bounded, nobody forgot anything, and they got the bill anyway.
The cap is a fuel gauge, not a progress bar
Every agent framework stops the same way, and I mean every one, I read the source of nine of them. The loop continues until the model stops asking to continue, which is the model deciding for itself whether it is done, plus a counter, a max-iteration or a recursion limit. That is the entire architecture: the model's opinion, and a counter.
The counter is a real, valid termination proof. Max iterations minus the count goes down by one each pass and bottoms out at zero, so the loop provably stops, eventually. So why did it run for 11 days? Because the counter is a fuel gauge, and everyone reads it as a progress bar.
A fuel gauge is accurate and honest. When it hits empty, the car stops, guaranteed. But two things stop a car: you arrived, or you ran out of gas, and the fuel gauge cannot tell you which. It measures fuel, and "did I get there" is a different question. The iteration cap measures how much you have spent, not how close the task is to done, and in the code a run that hit the cap and a run that produced a real answer come back as the same type, so the caller cannot tell them apart. "Stopped because I gave up" and "stopped because I am done" look identical, and every layer downstream believes the second one, and writes it to a database, or sends it to a customer, or feeds it to the next agent.
Here is the quick way to prove the cap is doing nothing: delete it and rerun. In a healthy system, deleting a load-bearing variable breaks it, the way deleting the congestion window from TCP causes the internet to fall over, which actually happened in 1986. Delete the iteration cap from your agent and it makes the same tool calls in the same order, just for a bit longer. The trace does not flinch. The cap is scaffolding, not a real control. It bounds the wrong thing.
Every stopping rule needs a model of the task
This is the center of it. Every field that ever built a reliable automatic stopping rule needed a model of the task itself. Munitions inspection in 1945, early stopping in clinical trials, Bayesian optimization, multi-armed bandits, proof assistants that will not compile until you supply the measure yourself. Eighty years, at least seven fields that do not read each other's papers, and every one needed a number that describes the work, not the effort. That is not a coincidence, it is a requirement.
Your LLM agent has a model of language and no model of the task. That is the direct consequence of the thing that makes it general, and it is why the stopping rule is missing. The framework authors are not being lazy, they are attempting the general case that nobody has solved, so they honestly shipped you a counter and left the real measure to you, because only you can define what "done" means for your task.
The failure your monitoring cannot see
Now the expensive one. Picture a planner and an executor, each with its own progress measure. The planner makes a plan, its measure descends, it is doing its job. The executor works the plan, its measure descends, it is doing its job. Then the planner replans, which is exactly what it is designed to do, and that resets the executor's measure back to the top. The executor works the new plan down again, the planner replans again, and the executor's measure resets again.
- Planner measure descends 5 to 3healthy
- Executor measure descends 4 to 2healthy
- Planner replansnormal
- The replan resets the executor measure back to 4each component healthy, the composition never terminates
- And the cycle repeats
Neither component is doing anything wrong. Each measure is a genuine, valid termination proof on its own. But composed in one system, every step passes and the whole thing hangs forever, because the mathematical fact is that two correct termination proofs do not compose into a correct one, and there is nowhere you can stand inside either component and see it. On the dashboard, the planner panel is healthy and descending, the executor panel is healthy and descending, and the system has burned 47,000 dollars over 11 days, and both panels are correct. Nobody is lying to you, and the alert never fires, because nothing is wrong with either component on its own. This is the failure your monitoring is structurally incapable of seeing, and no amount of per-component instrumentation will catch it, because the problem is in how they interact.
A bound on a loop is not a bound on a system of loops
So where did 47,000 dollars come from if every run is capped at about 100?
A call tree. Each node is an agent run, each capped at 25 iterations, each provably bounded.
Depth one is 25 calls. Depth two is 625. Depth three is over 15,000. Depth four is over 390,000. Every single node is individually and provably bounded, and nothing anywhere bounds the depth.
A bound on a loop is not a bound on a system of loops. Every call was bounded. The system was not.
The agent reliability calculator prints this tree on your own branching factor, depth, and cost per run.
And it never tripped the rate limit, because the rate limiter watches requests per minute, and this was never fast. It was endless. Endless and slow is a different failure from fast and abusive, and your rate limiter only knows about the second one.
What to instrument instead
Per-component instrumentation cannot see either the composition hang or the call tree, because both are properties of the whole. So the instruments have to live at the whole.
Carry a task identifier through every nested call, so that a tree of runs is recognizable as one task rather than as four hundred unrelated ones. Without it, nothing in your telemetry knows these calls belong together, which is exactly why nobody noticed.
Then bound the task, not the call. A wall-clock deadline on the entire task, a spend budget on the entire task, and a maximum depth, all enforced by something outside any individual agent and all inherited by every child call. A child that cannot see the parent's remaining budget cannot help you spend it responsibly.
And alert on aggregates rather than on instances. Spend per task, wall-clock per task, depth reached, and calls per task. Every one of those was screaming in the example above. Nothing was watching any of them, because every dashboard panel was scoped to a single run, and a single run always looked fine.
The missing third state
Every mature field has a state your agent is missing.
Chess has a draw, which is not a loss. It is its own outcome, with its own rules and its own meaning. A constraint solver returns "unknown," which means "I ran out of budget," and is carefully distinguished from "no answer exists," because those two facts lead to opposite next actions. A database names a deadlock victim, so the losing transaction knows it was killed rather than that it failed, and therefore knows that retrying is reasonable.
Your agent has two states: done, and killed. It is missing the third one, gave up.
That missing state is the whole gap. When the cap trips, the run still has to return something, and it can only be filed as one of the two states the system has a name for. So it gets filed as done.
This has to be fixed at the return type, not in a log line. If "gave up" is a field inside a success response, every caller that does not specifically check that field will treat it as success, and most callers will not check, because the type told them they did not have to. Make the three outcomes structurally distinguishable so that a caller cannot accidentally ignore the difference, and so that a new caller written a year from now has to handle it.
The downstream consequences differ completely. A done result can be stored, sent, and built upon. A gave-up result should be retried with more budget, escalated, or surfaced as incomplete, and it must never be silently persisted as an answer. A killed result usually means something external intervened and the state may be inconsistent. One return type cannot carry three meanings.
The fix, and the decision only you can make
The decision is yours, not your engineers': does your task have a computable residual? Is there a number you can calculate in code, without asking the model, that gets smaller as the task gets closer to done?
For many tasks there is one sitting in plain sight. A migration has rows not yet migrated. A reconciliation has records still unmatched. A test-fixing loop has failing tests remaining. A form-filling task has required fields still empty. A document pipeline has pages not yet processed. Each of those is a real number, computed by your code, that the model cannot talk its way past. If you have one, put it in the guard, and this is roughly three days of work rather than a project.
For some tasks there is no natural number, and then the question is whether you can manufacture one with a verifier or a rubric that a machine can evaluate. That is the real engineering, and it is where the effort should go.
And if you genuinely cannot, the honest conclusion is that you should not ship an autonomous loop for that task at all. Ship a bounded assistant with a human as the terminal state. That is not a failure of ambition, it is the correct read of what the task supports, and it is much cheaper than discovering the same thing from an invoice.
Concretely, five moves for your next design review:
- Name the number that goes down, and put it in the guard, in the code, not in a prompt.
- Demote the model to a proposer. It suggests the next move, it does not get to certify its own completion, the same way a fuel gauge is a float in the tank, not the engine's opinion of how much fuel is left.
- Give the system three terminal states that are distinguishable at the return type: done, gave up, killed.
- Add a time fuse for hangs and an iteration fuse for spins. These are different instruments, and you probably have two of the same one. A hung tool call freezes the counter forever, so only a clock, on separate hardware, catches it, which is exactly why every embedded watchdog counts time, not iterations.
- Bound the system, not the loop. Find every place two capped things call each other.
There is a precise version of this one layer down, in the network stack you are reading this over. TCP's receive window was a real, correctly enforced bound, and congestion collapse happened anyway, because it bounded the wrong thing, the receiver instead of the network in the middle. The fix in 1988 was to add a second window driven by observed loss and take the minimum of the two. Your iteration cap is that receive window: real, correctly enforced, bounding the wrong thing. You do not delete it, you add the measure that describes the task and take the minimum.
Two honest limits. A proof that your agent "terminates almost surely" is a theorem about eternity, not your invoice, because the expected time and the expected cost can both be infinite, so ask anyone who shows you one which they actually proved. And if your measure hits zero while the task is still half undone, your measure was a proxy that bottomed out early, all tests pass and the feature is still wrong, and nothing here closes that gap, because manufacturing a faithful residual is a genuine research problem. The real problem was never "make the agent stop," it was "give it a measure of the task," and that is where the actual engineering is.
FAQ
Why does my AI agent keep running and not stop? Because its only stopping signals are the model deciding it is done and a counter that measures spend, not progress. Neither describes the task, so the loop keeps going while the fuel gauge slowly drains, and a hung tool call or two loops feeding each other can keep it running indefinitely.
Does setting a max iteration or timeout stop a runaway agent? It bounds a single run, but it does not stop the system. The cap measures effort, not remaining work, so it cannot tell "done" from "gave up," and a tree of individually capped runs can still cost thousands, because a bound on a loop is not a bound on a system of loops.
Why did my agent bill thousands of dollars with no error? Because every call was individually bounded while nothing bounded the whole system, most often a call tree of capped runs or two components that keep resetting each other's progress. Both look healthy on a dashboard, so no alert fires, and the bill grows quietly.
How do I detect a runaway agent before the invoice? Carry one task identifier through every nested call so a tree of runs is recognizable as a single task, then bound and alert on the task rather than the call: spend per task, wall-clock per task, depth reached, and calls per task. Per-run dashboards cannot see this, because every individual run looks fine.
What is a computable residual for an agent task? A number your code can calculate without asking the model, that shrinks as the task nears completion: rows not yet migrated, records still unmatched, tests still failing, required fields still empty. If your task has one, it belongs in the loop guard. If it has none and you cannot manufacture one, do not ship an autonomous loop for that task.
How do I make an AI agent stop reliably? Put a number in the guard that is computed in code, without the model, and that shrinks as the task nears done. Demote the model to a proposer, give the system three terminal states (done, gave up, killed), use separate fuses for hangs and spins, and bound the whole system, not each loop.
Tell us the system, the stakes, and the date that matters. You get a straight technical reply from the person who would lead the work, within 24 hours.
Bring us the program