Tagged diagnose
12 pieces
- · 7 min read
Perception to Control: The Handoff Nobody Owns
The perception team ships a detector that meets its accuracy target. The controls team builds a controller that is stable given its inputs. Both are right and the robot behaves badly, because what crosses between them is not a detection, it is a detection with a latency, a coordinate frame, a timestamp, and an uncertainty, and only one of those five is usually specified.
- · 7 min read
PLC to Model: Where Industrial AI Actually Breaks
The model is rarely the problem on a factory floor. The problem is the path between the controller and the model: a protocol implemented as the vendor felt like implementing it, a timing budget with the round-trip inside it, a controller that cannot be taken out of service, and a boundary that belongs to neither the automation team nor the data team.
- · 7 min read
Signs the Failure Is in the Seams, Not the Model
Every component passes its own tests. Every team's investigation ends politely at their boundary. The system does not work. Five signs tell you the failure lives between components rather than inside one, and they matter because a seam failure bought as a component engagement is money spent on the wrong question.
- · 7 min read
What a Two-Week Diagnostic Should Deliver
A diagnostic is not a discovery phase, an architecture review, or a slide deck of observations. It is a finding with evidence attached, a ranked list of what else is wrong, and a scoped fix you could hand to a different vendor. If the deliverable cannot be handed to someone else, it was a sales document.
- · 7 min read
How to Tell Your Vendor Is Aiming at the Wrong Thing
The expensive failure is almost never a vendor who cannot execute. It is a competent team executing precisely, on the wrong component. That is much harder to detect, because everything looks like progress, and there are four tells that show up before any code does.
- · 8 min read
What It Costs to Fix a Failing AI System
Nobody in this field answers the cost question straight, so here it is. The price is not set by your system's complexity, it is set by which of four diagnoses you land on, and the four are separated by more than an order of magnitude. Any quote produced before the diagnosis is a quote for the most expensive branch, because that is the only one you can price without looking.
- · 7 min read
The Five Ways a Loop Fails to Stop
Floyd gave us the conditions a terminating loop has to satisfy in 1967. Negate them one at a time and you get an exhaustive list of the ways termination can fail, which turns a vague symptom into five named diagnoses with five different fixes. Your agent is almost certainly number four, and number four is the only one that looks like success.
- · 7 min read
The Delete-the-Variable Test
A ten-second check that tells you whether a variable is doing real work or is scaffolding: delete it, rerun, and see what changes. Delete TCP's congestion window and the internet falls over, which it did in 1986. Delete your agent's iteration cap and the trace does not flinch. The test has one important limit, and knowing it is what stops the test becoming a licence to remove your own safety nets.
- · 8 min read
Rebuild or Repair? How to Tell Before You Approve Six Months
Two contractors quoted four and six months to rebuild a model. The actual fix took five days and was not the model. The decision between rebuilding and repairing is not a judgement call, it is a diagnosis you can run in an afternoon, and running it first is the difference between a five-day fix and six figures spent reproducing the same failure.
- · 7 min read
Every Metric Is a Proxy: Goodhart's Law for AI Systems
You cannot read forty thousand answers, so you appoint one number as a messenger and send it to check the truth. Every dashboard number is that: a stand-in for something you actually care about but cannot watch directly. The gap between the number and the thing is where trouble hides, and the moment the number becomes the goal, people find ways to move it that have nothing to do with what you wanted.
- · 8 min read
Before You Blame the Model: The Four Boring Checks
When an AI system works in testing and fails in production, the model is the last thing to suspect, not the first. Four checks rule out the plumbing before you touch anything interesting: is the same model actually running, is it getting the same input, is the preprocessing identical, and is something quietly timing out or falling back. Most reported model failures die at one of these four.
- · 15 min read
Why Your AI Works in Testing but Fails in Production
When a model scores 95% in staging and collapses to 44% in production, the model is almost never the cause. The usual fix, rebuilding the model, retrains on the same data and reproduces the same failure. The real problem is what the model learned from the data, and it is usually fixable in days, not months.