Negative Controls: Prove Your Check Can Fail Before You Trust Its Green

Mostafa Dhouib·August 28, 2026·6 min read

Negative Controls: Prove Your Check Can Fail Before You Trust Its Green

The short answer. Break the thing a check guards, confirm the check fails, then restore. Every incident in this pillar is a check that had never been shown to fail, and each one is caught by one of three cheap forms of the same move.

Where the term comes from

In a laboratory, a negative control is the sample that must come back negative. You run it alongside the real samples, every time.

If the negative control comes back positive, the finding is not that the sample was unusual. The finding is that your assay is wrong, and every other result in the batch is void. Nobody argues about this. It is not a nice-to-have, it is what makes the other results mean anything.

Software has exactly the same need and almost none of the habit. We run the positive case, get the expected result, and treat that as validation of the instrument as well as the sample.

Form one: inject a known defect

Put a deliberate fault into the module a test protects, and confirm the suite fails. Then revert.

This is a manual, one-off mutation, and it is the cheapest possible version of the discipline. It needs no tooling and no budget, and it takes minutes.

Two rules make it worth doing rather than performative.

Pick the code you would least like to be silently wrong, not the code that is easiest to reach. Money, safety, access control, or the boundary where you hand a value to somebody else. The value of the exercise is entirely determined by which line you choose.

Change behaviour, not structure. Deleting a log line proves nothing. Flip a comparison, change a constant, remove a guard clause. You want the program to do something different, so that a test asserting on behaviour has something to notice.

The systematic, automated version of this is mutation testing, which is the same idea run continuously rather than once.

Form two: keep a must-fail case in CI

Keep a known bad build or a known bad input in the pipeline whose job is to fail.

This is a meta test on the harness itself, and it is the only check in your entire system that watches the watcher. If it ever goes green, the harness is broken, and you learn that on the day it breaks rather than during the incident it was supposed to prevent.

It matters more than it sounds because harnesses break silently and in boring ways. A test runner upgrade changes how failures are reported. An exit code gets swallowed by a shell wrapper. A CI config change makes a job non-blocking. A refactor moves the assertions into a file the glob no longer matches. In every one of those cases the pipeline goes green, faster than usual, and nobody investigates a fast green build.

One deliberately broken case catches all of them.

Form three: drive the out of range condition

For alarms, guards and limits, physically produce the condition and confirm the alarm fires.

The question is embarrassingly simple and almost nobody has answered it: does the alarm actually fire?

Not: is the alarm configured. Not: does the alarm appear in the runbook. Has anybody, ever, driven the input past the threshold and watched it go off.

This is the direct analog of the two organisational failures in this pillar. MCAS's response to a bad sensor was recorded as acceptable and never driven. Knight Capital's 97 warning emails did fire, into a channel where firing meant nothing. Both are alarms whose behaviour was assumed rather than observed, at opposite ends of the failure.

In firmware this generalises into fault injection: corrupt a sensor value, force an error return, flip a bit, drop a packet, delay an interrupt. Same move, more inputs.

Which incident each form catches

Worth laying out, because the mapping is close to one to one.

CrowdStrike: a field count assertion in the validator, or one test with a non-wildcard twenty first field. Form one, applied to a validator.

Heartbleed: a malformed heartbeat with declared length far greater than actual. Form three, applied to a parser.

Knight Capital: a version consistency check, and then the negative control on that check, which is deliberately skipping a node in staging and confirming the deploy fails. Note the recursion: the new control needs its own negative control, or it is just another unproven green.

Mars Climate Orbiter: a units assertion at the contractor boundary. Form one, applied at an interface rather than inside a module.

MCAS: fault injection with a single erroneous sensor value, at the system level. Form three, at the altitude where the assumption actually lived.

The recursion is the point

Notice what happened with Knight. The fix is a version consistency check. But a version consistency check is a check, and everything in this pillar says a check nobody has watched fail is not evidence.

So the fix has two parts, and the second is the one that gets skipped: build the control, then break something and watch it fire.

That is not paranoia and it is not infinite regress. It terminates immediately, because you only need to observe each check fail once to convert it from a belief into an observation. The cost is minutes per control, one time.

The general argument is in a test that has never failed is a decoration. To score which of your own controls have actually been watched failing, with that distinction built into the scoring, use the verification diligence scorecard.

FAQ

What is a negative control in software testing? Deliberately breaking the thing a check guards and confirming the check fails, then restoring. The term comes from laboratory practice, where a sample that must come back negative validates the assay rather than the sample.

How do I test that my tests work? Three forms: inject a known defect into protected code and confirm the suite fails, keep a must-fail case in CI as a meta test on the harness, and drive out of range conditions on alarms to confirm they fire.

What is a must-fail test in CI? A known bad build or input kept permanently in the pipeline whose job is to fail. If it ever passes, the harness itself is broken, which is otherwise a silent failure that makes every other green meaningless.

Which line should I break first? The one you would least like to be silently wrong: money, safety, access control, or a boundary where you hand a value to another team. And change behaviour rather than structure, so that a behavioural assertion has something to notice.

Is this the same as mutation testing? Mutation testing is the systematic, automated form of the first of the three. The manual version needs no tooling and is the right place to start.