Reference

Every number, with what it actually supports.

Figures get quoted without their context, which is how a measurement becomes a claim it never made. So here is each one with the conditions it was measured under, and an honest note on what it does not show.

Reliability compounding

Arithmetic rather than measurement. Reproducible by anyone with a calculator, and the assumption of independent step failures makes every figure an optimistic bound.

77%End-to-end success across 5 steps at 95 percent each

Every step passes its own test. The loss is in the connections.

Why your AI agents fail

36%End-to-end success across 20 steps at 95 percent each

Model and prompt unchanged. The only variable is the number of steps.

Why your AI agents fail

4%End-to-end success across 40 steps at 95 percent each

A real production system, failing 96 percent of the time.

Why your AI agents fail

99.7%Per-step reliability required for 95 percent across 20 steps

Not 95 percent. Roughly two orders of magnitude apart on the error rate.

Agent reliability calculator

25 / 625 / 15,625Runs at depth one, two and three in a call tree branching 25 ways

Every node individually and provably bounded. Nothing bounds the depth.

Why your agent will not stop

Measured in real engagements

Figures from systems worked on directly. Clients are anonymised and some values are shifted enough to prevent identification, never enough to change what they demonstrate.

95% to 44%Accuracy in staging against accuracy a week after launch

A speech command system. One person had recorded nearly all the training data.

Why your AI works in testing

5 daysTime to recover that system to above 99 percent

Balanced the data, added real-world noise, retrained the same model. Two contractors had quoted four and six months to rebuild.

Why your AI works in testing

88% / 41%Average retrieval accuracy against accuracy on scanned PDFs

The failing slice was small enough not to move the average, and a disaster for the people in it.

How to fix a broken RAG system

82%Reduction in cascading failure across an agent chain

Same model, same prompts. Each step was given the ability to reject a bad input.

How one bad input poisons a chain

85%Share of failures traced to one step of eleven, once instrumented

Two days to instrument, two days to fix. Before that it was called random.

Instrumenting an AI agent

$47,000 / 264 hoursSpend and runtime of an agent that never stopped

Status green throughout, no errors, every individual call correctly bounded.

Why your agent will not stop

1.6x overCurrent the reference amplifier had to sink against what it could

0.8 microamps required, about 0.5 available, every component at nominal. Architecture rather than margin.

The return-path bug

2.4 M to 600 kCurrent limiting when two correct resistors are placed in parallel instead of series

Passes every part inspection, because both resistors are the right value.

Topology vs parts

From published research and public record

Not our measurements. Included because they corroborate the pattern from outside.

40 / 37 / 21Percent of multi-agent failures from specification, coordination, and missing verification

Categorised across hundreds of execution traces. None attributed to the model.

Why your AI agents fail

$440M / ~5 minLoss from an automated trading deployment with no kill switch, 2012

The code executed. The system had no way to stop it.

Designing for irreversibility

~50 millionPeople without power in the 2003 northeast blackout

A software fault stalled the control-room alarms. Operators flew blind while the grid cascaded.

Instrumenting an AI agent

1986Year congestion collapse dropped a link from 32 kbit/s to 40 bit/s

The receive window was real and correctly enforced. It bounded the wrong thing.

Why your agent will not stop

How to read these

The compounding figures are arithmetic and you should reproduce them rather than trust them. The engagement figures are single cases, not benchmarks: they show that a failure had a cheap cause in one real system, not that yours does. The third group is other people's work, included because it corroborates the pattern from outside rather than because it is ours.

Nothing here is a projection of what a system would achieve. Where a figure is a design budget rather than a measured result, it is labelled as one in the article it comes from.