Nodes spawn children with some probability rather than always, which is what makes the outcome a distribution instead of a number. Take the branching factor and the spawn rate from traces if you have them; if you do not, that absence is worth noticing, because it means nothing is bounding the tree today.
| p50 | p90 | p99 | Worst of 1,500 | |
|---|---|---|---|---|
| Nodes in the tree | 18 | 42 | 69 | 95 |
| Spend | $1 | $3 | $4 | $6 |
| Wall clock | 3.9 min | 9.4 min | 15 min | 23 min |
Nodes in a fully expanded tree
The arithmetic everybody does, and the reason the numbers feel unreal. This is every node spawning the full branching factor to full depth, which almost never happens. It is the right bound and the wrong planning number, and its purpose here is to show how far above the realistic distribution it sits.
| Branching factor | Depth 2 | Depth 3 | Depth 4 | Depth 5 | Depth 6 |
|---|---|---|---|---|---|
| 2 | 3 | 7 | 15 | 31 | 63 |
| 3 | 4 | 13 | 40 | 121 | 364 |
| 4 | 5 | 21 | 85 | 341 | 1,365 |
| 5 | 6 | 31 | 156 | 781 | 3,906 |
Amber is over 100 nodes, red over 1,000. At $0.06 a call with two retries available, the depth-5 branching-4 cell is 341 nodes and up to $61 for one user action. The simulator above will tell you that the realistic p99 is a fraction of that, which is the point of running it.
A worked example
A research assistant where a node spawns sub-tasks about seventy percent of the time, roughly three at a time, to a depth of four. Calls take about nine seconds at the median with a fat tail, cost six cents, and fail seven percent of the time with two retries available. The team has capped each agent at twenty iterations and considers the cost bounded.
The median task is 18 nodes and about $1.14, which is comfortable and is the number anyone sampling a few runs would report. The p99 is 69 nodes and $4.32, nearly four times the median, and the worst of fifteen hundred simulated tasks is 95 nodes.
Wall clock is where it bites. The median task finishes in 3.9 minutes and the p99 takes 14.6 minutes, because the latency tail compounds down every level of a serial tree. Against a 12-minute deadline, 4.1 percent of runs are cut off while still working.
The per-agent cap never fires in any of this. It bounds a node, and nothing in the system bounds the tree, so the only thing standing between this and a much worse day is that the spawn rate has not drifted upward. The fix is a task-level budget carried through every nested call, set from the p99 rather than from the maximum.
The arithmetic, so you can check it
Each node retries while a draw falls under the failure rate, up to the retry limit, and every attempt costs and takes time. Latency is drawn lognormal from your median and spread, because latency is right-skewed and a normal draw understates the tail that causes the incident. A node spawns children with your spawn probability, and how many is itself a draw around the branching factor.
Under serial execution a parent waits for each child in turn, so subtree times add. Under parallel it waits for the slowest, so the wall clock is the critical path while the spend is unchanged. The simulation is seeded from the inputs, which means the permalink carries the result rather than a different sample.
Every distribution here is a modelling choice. Real spawn behaviour is correlated with task difficulty rather than independent per node, so hard tasks branch more at every level and the true tail is heavier than this shows. Rate limits, concurrency ceilings and shared caches are not modelled at all. Treat the output as a lower bound on the tail and a good guide to shape, and instrument the real thing: the largest tree you have actually seen is a number worth having, and most teams do not have it.
The reasoning is in why your AI agents fail and bounded the wrong thing. For the chain reliability side, use the agent reliability calculator.
Questions
Because the caps multiply rather than add. Twenty capped agents each able to invoke twenty capped agents is four hundred runs, and every one of them stayed inside its own limit. Nothing was violated, nothing alerted, and the bill arrives anyway. A tree needs a budget carried through every nested call and enforced outside any individual agent.
Because the worst case is the wrong number to plan against. Full branching to full depth with every retry taken is a bound so far above what happens that a budget set from it never binds, which means it protects nothing. The p99 is the number that binds on runs you will actually see, and getting it requires knowing the distribution rather than the maximum.
The p99 of the simulated distribution, accepting that roughly one run in a hundred will be stopped by it. Setting it from the mean means being over budget constantly, and setting it from the theoretical maximum means having no effective limit at all. The choice is which of those two errors you prefer, and the p99 is usually the honest middle.
No. Parallelism changes wall clock, not spend: the same nodes execute either way. It also changes which failure you meet first, because a parallel tree hits rate limits and concurrency ceilings that a serial one never approaches, while a serial tree hits the deadline first.
From traces, by counting how many child invocations each node actually made across a sample of real tasks. If nobody can produce those numbers, that absence is the finding rather than an inconvenience, because it means nothing in the system is currently bounding the tree and nobody could tell you what the largest one has been.