Every Stopping Rule Needs a Model of the Task: Eighty Years of Evidence

Mostafa Dhouib·September 4, 2026·8 min read

Every Stopping Rule Needs a Model of the Task: Eighty Years of Evidence

The short answer. Every field that has ever built a reliable automatic stopping rule needed a model of the task itself, not of the effort spent on it. Sequential testing in 1945, early stopping in clinical trials, Bayesian optimization, multi-armed bandits, index policies, and proof assistants that refuse to compile until you supply the measure. Eighty years, at least seven fields that do not read each other's literature, and every one arrived at the same requirement. Your agent has a model of language and no model of the task, which is not an oversight. It is the direct cost of the thing that makes it general.

There is a particular kind of evidence that is worth more than any single result: the same requirement discovered independently, repeatedly, by people who were not talking to each other.

That is what this is. And it explains why the stopping problem in agent systems is not a framework bug waiting for a patch.

What every one of them needed

Sequential analysis, 1945. Wald's sequential probability ratio test computes a log likelihood ratio and compares it against two boundaries. That is the stopping rule, and it came out of wartime munitions inspection, where testing every unit destroyed the inventory and testing too few shipped defective ordnance. What it needed: a parametric model of the observation process. It knew what the data-generating process looked like.

Clinical trials. Trials stop early for futility by computing conditional power against a threshold. This is not a convenience. Stopping a trial that cannot succeed is how people stop being enrolled into something that will not help them. What it needed: a model of the trend and of the patients remaining.

Bayesian optimization. Expected improvement weighed against the cost of the next evaluation, a formulation from 1978 that gives a cost-aware stopping rule with a guarantee. What it needed: a surrogate model of the objective, handed to it on day one.

Multi-armed bandits. Confidence bounds against a separation criterion. What it needed: a reward distribution per arm.

Index policies. The Gittins index, 1979, computes an index per arm against the others. What it needed: independent Markovian arms, which is to say a model of how each option evolves.

Proof assistants. A dependently typed language will not compile a recursive function until you supply the decreasing measure yourself. What it needed: you, explicitly, because the system refuses to guess.

Six of the seven derived the measure from a model of the task. The seventh refuses to proceed without one. None of them found a way around it.

Now put your agent in that table

Your agent's continue condition computes nothing about the task and compares against nothing that describes it. It checks two things: whether the model asked to continue, and whether a counter has run out.

The model has a model of language. It has no model of the task.

That is the general statement, and it is the intellectual centre of the whole problem: every computable stopping rule needs a model of the task. Eighty years, seven fields, no exceptions, and none of them talking to each other. That is not a coincidence and it is not an implementation detail. It is a requirement.

Why this is not the framework authors being lazy

It would be easy to read the above as an indictment. It is not, and getting this right changes what you do next.

Bayesian optimization got a cost-aware stopping rule with a guarantee in 1978, and it had a full surrogate model of the objective handed to it on day one. It knew the shape of the problem before it started.

Your agent loop has had roughly three years of serious attention, and it has no model of the task at all, because the task is different every time. That is not a gap in the framework. It is the definition of the thing: an agent that is general cannot come with a model of your specific task, because it does not know what your task is.

So the framework authors did the honest thing. They refused to ship a measure that does not exist, and gave you a counter instead. I would rather have a counter and know it is a counter than a fabricated progress metric I might believe.

Which means the answer to "should we wait for the frameworks to fix this?" is no, and not because they are slow. You are asking them for a stopping rule on a task you have not defined. Nobody can define it but you. It is not a roadmap item, it is a category error.

The decision this leaves you

The decision is yours, not your engineers'. It has exactly three branches.

Does your task have a computable residual? A number you can calculate in code, without asking the model, that gets smaller as the task gets closer to done.

For many tasks there is one sitting in plain sight and nobody looked. A migration has rows not yet migrated. A reconciliation has records still unmatched. A test-fixing loop has failing tests remaining. A form-filling task has required fields still empty. A document pipeline has pages not yet processed. A proof search has open obligations. Each is a real number your code computes and the model cannot talk its way past.

If you have one, put it in the guard, and this is about three days of work rather than a project.

Before that, it is worth knowing how far off you are: the agent reliability calculator gives the end-to-end number your current per-step figure produces.

If not, can you manufacture one? A verifier, a rubric, a generated test suite, a checkable property. This is the real engineering, and it is where the effort should go. It is also where I would put money if I were funding work in this area, because it is the actual open problem rather than the plumbing around it.

If you genuinely cannot, accept it. Do not ship an autonomous loop for that task. Ship a bounded assistant with a human as the terminal state. That is not a failure of ambition, it is a correct read of what the task supports, and it is considerably cheaper than learning the same thing from an invoice.

The property that makes a measure work

One more thing the seven fields share, and it is easy to miss.

In every one of them, the measure is computed by something other than the thing being measured. Wald's likelihood ratio is computed from the observations, not from the process's opinion of itself. The trial's conditional power comes from the data. The proof assistant makes you supply it.

The same move applies to your agent: demote the model to a proposer. It suggests the next action. It does not get to certify its own completion. This is the fuel gauge again, which is a float in the tank rather than the engine's opinion of how much fuel is left. The thing being measured does not get a vote on the measurement.

A measure the model can influence is not a measure. It is another opinion, and you already had one of those.

The honest limits

Two, and they matter enough to state.

A residual that reaches zero while the work is genuinely half done means your measure was a proxy that bottomed out early. All the tests pass and the feature is still wrong. Nothing here closes that gap, because manufacturing a faithful residual is a real research problem rather than an engineering one.

And a sufficient criterion is not a necessary one. A system with zero expected drift can still terminate. So the absence of a decreasing measure does not prove your loop runs forever; it only means you cannot prove it stops, which is a different and more common situation.

FAQ

How do I make an AI agent terminate reliably? Put a number in the loop guard that your code computes without asking the model and that shrinks as the task nears completion. Rows unmigrated, tests still failing, records unmatched, required fields still empty. If your task has one, this is days of work rather than a project.

Why don't agent frameworks solve the stopping problem? Because a stopping rule requires a model of the task, and a general framework cannot have one, since it does not know what your task is. They ship a counter and are honest that it is a counter, which is better than fabricating a progress metric you might believe. The only fix possible is on your side.

What is a computable residual? A number calculated in code, deterministically and without a model call, that decreases as the task approaches done and has a floor it cannot go below. It is the same object that sequential analysis, clinical trials, Bayesian optimization, bandits, and proof assistants each needed, arrived at independently.

Why can't the model decide when it is done? Because the thing being measured does not get a vote on the measurement. Ask a model twice on identical state and you can get two different answers, which means it is not a function of state and cannot serve as a termination measure. Demote it to a proposer that suggests the next action without certifying completion.

What if my task has no natural stopping measure? Try to manufacture one with a verifier, rubric, or generated test suite, which is the real engineering here. If you genuinely cannot, do not ship an autonomous loop for that task. Ship a bounded assistant with a human as the terminal state.