Tool

What your per-step number actually delivers.

Reliability multiplies across steps, so a chain of individually good steps is a bad system. And a bound on a loop is not a bound on a system of loops. Both pieces of arithmetic are below, on your numbers.

Part one: the chain
Every model call, tool call, or decision the task passes through.
%
How often one step is correct. Be honest rather than aspirational.
%
What the whole task has to achieve to be shippable.
End to end, as built
35.8%
20 steps at 95.0% each
Per step required
99.7%
to reach 95% across 20 steps
Distance, on the error rate
19.5x
how many times your error rate has to fall
At 95.0 percent per step, a 20-step task finishes 35.8% of the time. To hit 95 percent, every step needs 99.7%, not 95.0 percent. That is 19.5 times less error per step, and no model release closes a gap that size. Near perfect is not the neighbour of good here.
This calculator runs entirely in your browser. Nothing you type is sent anywhere unless you ask for the result by email at the bottom of the page.
Part two: the call tree

Every run below is individually capped and provably bounded. Nothing here bounds the depth, which is the difference between a bound on a loop and a bound on a system of loops.

Often the iteration cap, since each iteration can call a sub-agent.
How many levels of runs calling runs. Capped at 6 here.
USD
What one capped run costs when it behaves.
DepthRuns at this levelSpend at this level
125$2,500
2625$62,500
315,625$1,562,500
Total runs in the tree
16,275
Worst-case spend
$1,627,500
against $100 for one bounded run
One run costs $100 and every run is bounded. The system is not: at depth 3 it reaches $1,627,500. It will not trip a rate limiter either, because this was never fast. It was endless, and endless is a failure your rate limiter does not know about.
Send me this result, with what I would do about it

Your inputs go with it. You get a straight technical reply naming the two or three things I would change first for a system with these numbers, from the person who would lead the work.

Your inputs are included so the reply can be specific.

The arithmetic, so you can check it

End to end. A task that passes through n steps, each correct with probability p, succeeds with probability p raised to the power of n. That is the whole formula, and it is unforgiving because it is multiplication rather than addition.

At 95 percent per step: five steps gives about 77 percent, eight gives about 60, twenty gives about 36, and forty gives about 4. The model does not change across that curve. The prompt does not change. The only variable is how many steps the task took, which is why a three-step demo tells you almost nothing about a forty-step production system.

What the target requires. Invert it. To reach a target t across n steps, each step needs t raised to the power of one over n. For 95 percent across twenty steps that is about 99.7 percent per step.

The reason that gap feels smaller than it is: 95 and 99.7 look adjacent, and on the error rate they are not. Five percent error against 0.3 percent error is roughly a factor of seventeen. For this kind of system, near perfect is not the neighbour of good. Near perfect is the neighbour of failure, and getting from one to the other is the actual engineering.

The call tree. If a run can spawn b further runs and that nests to depth d, the number of runs at each level is b to the power of that level, and the total is the sum across levels. With b of 25, depth one is 25 runs, depth two is 625, and depth three is over 15,000. Every single node is individually and provably bounded. Nothing bounds the depth.

That is the difference between a bound on a loop and a bound on a system of loops, and it is how a system where one run costs about a hundred dollars produces a five-figure invoice without a single call misbehaving.

The honest limit

This assumes steps fail independently, and real systems are worse than that rather than better. Errors inherit: one wrong assumption early becomes the foundation every later step reasons correctly on top of, so the failures correlate in exactly the direction that makes the real number lower. Treat the output as an optimistic bound.

What to do with the number

If the end-to-end figure is below your target, the lever is not a better model. It is fewer steps, or verification between steps so an error cannot propagate, or a step that can reject a bad input rather than doing its best with it. In one real system that last change alone cut the cascading failure by 82 percent, with the same model and the same prompts.

If the call-tree figure is the alarming one, the fix is to carry one task identifier through every nested call and bound the task rather than the call: a wall-clock deadline, a spend budget, and a maximum depth, all enforced outside any individual agent and inherited by every child.

The reasoning behind both halves is in why your AI agents fail and why your agent will not stop.

Questions

How do you calculate end-to-end reliability for a multi-step AI agent?

Raise the per-step reliability to the power of the number of steps. At 95 percent per step a five-step chain delivers about 77 percent, a twenty-step task about 36 percent, and a forty-step task about 4 percent, with the model and the prompt unchanged across all of them.

What per-step reliability do I need for a 20-step agent to work 95 percent of the time?

About 99.7 percent, not 95 percent. Take the target end-to-end reliability to the power of one over the number of steps. The gap between 95 and 99.7 percent is roughly two orders of magnitude on the error rate, which is why no model release closes it.

Why did my agent cost thousands of dollars when every run was capped?

Because a bound on a loop is not a bound on a system of loops. If each run can spawn further runs, the tree grows as the branching factor to the power of the depth, so twenty-five runs at depth one becomes 625 at depth two and over 15,000 at depth three, while every individual node remains provably bounded.

Why doesn't a rate limiter catch a runaway agent?

A rate limiter bounds requests per minute, and a runaway call tree is usually not fast. It is endless. Slow and endless is a different failure from fast and abusive, and only the second one trips a rate limit.