Why Your AI Agents Fail, and How to Fix It

Mostafa Dhouib·September 3, 2026·13 min read

Why Your AI Agents Fail, and How to Fix It

The short answer. An agent that demos beautifully and falls apart in production almost never has a model problem. The reliability you lose does not live inside any single step, it lives in the connections between them, and simple arithmetic makes it brutal: at 95% per step, a 20-step agent finishes far less than half the time. A better model makes it worse, not better. The fix is engineering the system around the model: verification, state, and guardrails.

I am the founder and principal engineer at Opulion, and over the last decade I have built and been called in to fix enough AI-powered systems to see the same failure happen over and over, different company, different stack, different industry, the same error every time. There is a story everyone tells about why these systems fail, and it is almost always wrong. Believing it costs months and a lot of money. So here is what actually breaks, from inside real systems, with no theory.

Get out of the code: the airport

Forget the code for a second. Picture an airport. The plane is your model, the AI, the pilot. Skilled, capable, expensive, and the part everyone obsesses over. And almost never the reason your flight is delayed.

Think about the last time you sat on a runway for two hours going nowhere. Who did you get mad at? The pilot, the airline, whoever was in front of you. The instinct feels right and it is completely wrong, because the pilot never touched the thing that delayed you. Your delay was born somewhere you cannot see, maybe three states away, inside air traffic control, in the coordination layer tracking a thousand planes and a thousand handoffs at once. Your pilot has zero control over any of it.

That is the whole talk in one sentence: you are blaming the visible part of a failure that lives in the coordination. Teams that spend weeks swapping models are swapping out a perfectly good pilot while the control tower is on fire.

The math nobody does

Here is the part nobody sits down and calculates, and once you see it you cannot unsee it.

Say each step of your agent works 95% of the time. You hear 95% and your brain files it under reliable, an A grade, ship it. Now watch what happens when you chain the steps. Step one, 95%. Step two, 95%. Five steps, each one 95% and each one passing its own test. Chained together they deliver 77%. Not one step was below 95%, and the system is at 77%.

The reliability you lost did not live in any one step. You can audit every step and every one comes back green. It lived in the connections between them. That is why most teams cannot find it: they profile the model, they test each component, everything comes back green, and the system is still broken, because the failure happens between the parts, not inside them.

And it gets worse the longer the task runs.

The model did not change across that curve. The prompt did not change. The only variable is how many steps the task took. So when someone tells you the demo was mind-blowing but production is garbage, the demo was not lying about the model. It was lying about how short it was. It showed you three steps and let you assume forty would behave the same way, and forty is a completely different universe. That gap is not a quality problem. It is arithmetic.

Now flip it. You want a real 20-step agent to work 95% of the time end to end. What does each step need to hit? Not 95%. It needs 99.7%, 997 times out of a thousand, no exceptions. And 95% per step feels like you are almost there, needing one more tweak. You are not. You are off by two orders of magnitude on the error rate. For this kind of system, near perfect is not the neighbor of good, near perfect is the same as failure, and getting from 95% to 99.7% is the real engineering. It is not prompt tweaking.

If you want the number for your own system, the agent reliability calculator does both halves of this arithmetic: what your per-step figure delivers end to end, and what your target actually requires per step.

The five ways agents actually fail

After doing this across a lot of systems, there are exactly five, and every one is a real system I was pulled into.

One, they multiply. Small errors compound across steps. That is the 77%.

Two, they inherit. One early mistake becomes the foundation for everything after it. In one real system, one step missed a single document. That wrong assumption got handed to the planning step, which generated three subtasks that each inherited it, and their outputs became the next step's input. Five steps down, the output had nothing to do with reality, confident and completely wrong. Every step was flawless, the model reasoned perfectly from beginning to end, but it was poisoned from a certain point, and every step downstream treated the garbage as settled fact. We did not touch the model. We built the system so every step could reject a bad input instead of swallowing it. Same model, same prompts, different system, an 82% reduction in the cascading failure.

Three, they can't be undone. A chatbot writes words, and if it is wrong you delete it, no harm done. An agent acts. It sends the email, charges the card, deletes the record, places the order, and those actions only turn one way, at machine speed, faster than you can reach in and hit stop. In 2012, an automated trading system shipped a bad deployment with no kill switch and lost 440 million dollars in about five minutes, and effectively ended a company that had been around for years. The code was the last domino. The failure was the system that let it run with no brakes. That is not a code problem, it is a systems problem.

Four, they can't be seen. A real system had 11 internal steps and logged exactly one thing, did the whole thing work, yes or no. All 11 steps in between were dark. So when it failed, it looked random, and I keep hearing good teams say it just fails randomly, we cannot reproduce it. There is no such thing as random failure, there is only failure you cannot see. The randomness is not in the system, it is in your visibility. We turned the lights on and instrumented every step, and 85% of the failures came from one specific step. Two days to find it, two days to fix it. It was never an unreliable system, it was one broken piece hiding in plain sight because nobody was watching the middle. This is old: in the 2003 blackout that cut power to 50 million people, a software bug quietly silenced the control-room alarms, and the operators flew blind while the grid cascaded around them. It was not the failure that did the damage, it was the view of the failure going dark.

There is a mechanism underneath that, and it is worth understanding because it explains why the failure is undetectable rather than merely missed.

Look at a correct step and a wrong step side by side in a trace. The correct one is confident, cleanly formatted, and carries a citation. The wrong one is identical. Same confidence, same formatting, same citation.

The wrong answer wears the exact uniform of the right one.

Here is why. At the moment the model produced that step, it may have been 51 percent sure or 99 percent sure. That difference existed, briefly, inside the computation. And then the step got written down as text, and the uncertainty was destroyed in the act of writing, because prose has no field for it. The page looks the same either way.

So the one thing you were counting on to catch this, the instinct that something looks off, never fires. Nothing looks off. It is not wrong in a way that looks wrong. It is wrong while looking exactly right, and every downstream step reads it as settled fact.

That is also why adding a human reviewer to the end of the chain catches so much less than people expect. The reviewer is reading the same uniform you are.

Five, they chase the wrong goal. They optimize exactly what you accidentally told them to, which is never quite what you meant.

This is not just my war stories. Researchers studied how multi-agent systems fail across hundreds of execution traces and categorized every failure: about 40% from bad specification and design, about 37% from coordination and handoffs, about 21% from nobody verifying the work. Specification, coordination, verification, that is the whole pie, and none of it is the model's fault.

Why "use a better model" is worse than useless

When your system fails, you reach for a bigger, newer, smarter model. It feels responsible. It is the trap. Everyone swaps the model and keeps the loop open, reason, act, reason, act, and nothing in that loop ever gets checked, so the system never knows if the last step was right and just keeps trusting itself until it fails badly.

A smarter model dropped into the same open loop just argues more convincingly from the same wrong starting point. You do not get fewer failures, you get more articulate, more confident, more expensive failures. The mistake gets better dressing. It is treating the patient before running the scan. It feels like progress the whole time, and the fix was never a bigger brain. It was checking between the steps and closing the loop.

What actually works: five system fixes

Notice that not one of these is "use a better model."

• See it. Record every step, so when something breaks the broken step is findable, not a mystery.
• Check it. Verify each step before the next one is allowed to trust it. Close the loop.
• Structure it. Let a step reject a bad handoff instead of swallowing it whole. That is the 82% fix.
• Contain it. Stop a failure at its source before it spreads through the chain.
• Bound it. Limit what the agent can do on its own. For critical, irreversible actions, put a human in the loop with a switch that can stop it before it causes real harm.

Each of those is more specific than it sounds, so here is what they mean in practice.

See it means a span per step with its inputs, its outputs, the decision taken, and a trace identifier threading the whole task together. The test of whether you have it: when a run fails, can you name the step that broke it in under ten minutes, without reproducing it? If not, you do not have observability, you have logs.

Check it means the next step does not get to assume the previous one was right. The check has to be independent of the thing it is checking, which rules out asking the same model whether it did well. Use a verifiable property, a second source, or a deterministic rule.

Structure it means a step can refuse a bad input and say so, rather than doing its best with garbage. Most agent frameworks make refusal awkward and completion easy, which is exactly backwards for reliability. This is the change that produced the 82 percent reduction.

Contain it means a failure stops at the step that produced it instead of propagating. In practice that is a boundary where a bad result becomes a task-level failure rather than an input to step six.

Bound it means enumerating what the agent can do without a human, and being deliberate rather than permissive about it. Sort actions into reversible and irreversible first, because that sort is where the unbounded downside lives.

Every item is the system around the model: orchestration, verification, state, guardrails. And here is the good news buried in it, and the most important thing in this article: this is buildable right now. You are not waiting for a lab to ship you a better model or a new framework. It is engineering, and you can start today. I know it works because I have done it repeatedly, and that 82% number was the same model and the same prompts, engineered properly. That was architecture, not intelligence.

This was never about AI

Zoom out, because this pattern was never about AI. The model is roughly 5%, the little box at the top. The pipelines, the protocols, the drivers, the firmware, the silicon, everything under it, that is the 95%, and that is where the failure lives. The exact same disease, one small error compounding through a system nobody is watching, shows up in systems with no model in them at all, a fleet of 25,000 servers talking over hardware protocols, a telemetry pipeline moving data off physical machines, protocol integration reaching down to the silicon. Zero AI, same failure, because it was never about AI. It was about production systems and the space between the parts.

That range is the point. Most firms live in one layer, the model, or the infrastructure, or the pipeline, and the failure almost always lives in the layer they never opened. That is the mechanical reason they cannot find what everyone else walks past.

What to take away

If you forget everything else: your model is probably fine, 99% of the time. The failure was almost never in the smart part. It was in the system around it, the coordination, the handoffs, the verification, the parts nobody is watching. The model is the pilot. The failure is in the airspace.

If you have a system that demos beautifully and falls apart in production, or you are about to build one, AI-powered or not, this is not a model problem and it is not a hardware problem. It is a systems problem, and a solvable one. Tell me the specific problem you are facing and I will write you back on your exact case.

FAQ

Why do AI agents work in a demo but fail in production? Because the demo was short. Reliability multiplies across steps, so at 95% per step a three-step demo runs at about 78% while a 20-step production task drops far below half. The model and the prompt are identical, the only variable is the number of steps.

Will a better or bigger model fix an unreliable agent? No, and it usually makes things worse. A smarter model in the same unchecked loop argues more convincingly from the same wrong starting point, producing more confident and more expensive failures. The fix is verification and guardrails between steps, not a better model.

How do I make a multi-step agent reliable? Engineer the system around the model: instrument every step so failures are findable, verify each step before the next trusts it, let a step reject a bad handoff, contain a failure at its source, and bound what the agent can do on its own with a human in the loop for irreversible actions.

Why does my agent fail randomly? It does not fail randomly, you cannot see it fail. A system that logs only a final pass or fail hides which of its internal steps broke. Instrument every step, and the randomness usually resolves to one specific step causing most of the failures.