Answer yes only where somebody has watched the check go red on a deliberately broken input. If the control exists and nobody has ever seen it fail, that is the middle answer, and it is the most common honest one. The distinction is the whole scorecard.
| Control | Weight | State | Precedent | The question they will ask |
|---|---|---|---|---|
| Every guard, alarm and limit has a test that drives the bad input | W5 | unproven | MCAS, 2018 to 2019 | Show me the test that supplies the out-of-range value and asserts the guard fires. |
| Deployments have a version-consistency check that goes red on divergence | W5 | absent | Knight Capital, 460 million dollars in 45 minutes | Skip one node deliberately in staging. Does the deploy fail? |
| Updates go out staged or canaried, never broadly at once | W5 | unproven | CrowdStrike, roughly 8.5 million machines | What is the largest blast radius a single content or config push can have? |
| Safety-relevant single points of failure are identified and fault-injected | W5 | unproven | MCAS, one angle-of-attack sensor | Which single inputs can take the system down, and when did you last corrupt one on purpose? |
| Every parser, protocol and sensor-input path has a fuzz test and a negative test | W5 | absent | Heartbleed, CVE-2014-0160 | Show me the test that sends a declared length larger than the actual payload. |
| Reused code is re-verified against the new operating envelope | W4 | unproven | Ariane 5, roughly 370 million dollars | Which modules came from another system, and what envelope were they verified against? |
| Units and interfaces are asserted at every team and contractor boundary | W4 | absent | Mars Climate Orbiter, a factor of 4.45 | Where is the assertion that the value crossing this boundary is in the units the spec requires? |
| The surviving-mutant list is reviewed on critical paths | W3 | absent | Google does not compute a mutation score | Show me the surviving mutants on your money or safety path, not your coverage number. |
| Flaky tests carry a ticket and a deadline, never a silent re-run | W3 | unproven | Google, 84 percent of pass-to-fail is flaky | How many tests are quarantined right now, and who owns each one? |
| Penetration test and audit findings are remediated, not filed | W3 | absent | Knight's 97 ignored warning emails | What was the last finding you closed, and how long did it sit open? |
| Verification effort is aimed where failure is most costly | W2 | unproven | Effort not tied to risk | Which part of the system would hurt most if it were silently wrong, and is that where the tests are? |
The twelve controls, and what each absence has cost somebody
Weight is how costly the absence has proven, not how hard the control is to build. That ordering is what makes these diligence questions rather than opinions about engineering taste: every row has a primary report behind it.
| Control | Weight | The precedent for its absence |
|---|---|---|
| Guards and alarms have a test that drives the bad input | W5 | MCAS, 2018 to 2019, 346 fatalities |
| Version consistency check on every deploy | W5 | Knight Capital, 460 million dollars in 45 minutes |
| Staged or canary rollout for every update | W5 | CrowdStrike, roughly 8.5 million machines |
| Single points of failure identified and fault injected | W5 | MCAS, one angle-of-attack sensor |
| Fuzz and negative tests on every input path | W5 | Heartbleed, CVE-2014-0160 |
| A must-fail case in CI, as a meta-test on the harness | W4 | The only check that watches the watcher |
| Reused code re-verified to the new envelope | W4 | Ariane 5, roughly 370 million dollars |
| Units asserted at every team and contractor boundary | W4 | Mars Climate Orbiter, a factor of 4.45 |
| Surviving mutant list reviewed on critical paths | W3 | Google does not compute a mutation score |
| Flaky tests carry a ticket and a deadline | W3 | Google, 84 percent of pass-to-fail is flaky |
| Audit and pentest findings remediated, not filed | W3 | Knight's 97 ignored warning emails |
| Verification aimed where failure is most costly | W2 | Effort not tied to risk |
The five weight-five controls are the cheapest things on this list to build and the most expensive to be without. Every one of them would have been a few days of work at the company it later cost nine figures or lives.
The state this usually finds
The scorecard opens on a realistic profile rather than an empty one, and it lands at 33 percent. What produces that number is not a team who skipped testing. It is a team with one control they have actually watched fail, six that exist and have never been observed firing, and five that are absent.
The six in the middle column are the finding. Guards that exist, a canary process on paper, single points of failure that are known but never deliberately corrupted, reused modules nobody re-verified. Every one of those would be reported upward as done, and none of them is evidence.
Two of the five absent ones carry a nine figure precedent: the deploy consistency check that would have caught Knight Capital, and the negative tests on input paths that would have caught Heartbleed. Both are days of work.
The fastest move before a raise or a diligence process is not to build the missing five. It is to watch four of the existing six fail, on camera, and keep the recording. That converts six beliefs into four observations in an afternoon, and it is exactly what a diligence engineer is going to ask you to do in the room anyway.
How it is scored
Each control scores its full weight when somebody has watched the check fail, half when the control exists but has never been observed firing, and nothing when it is absent. The score is earned weight over total weight, which is 48.
Gaps are ranked by weight rather than by how easy they are, and unproven controls are listed alongside absent ones, because until a check is watched failing the two are indistinguishable from outside.
The weights are a judgement informed by what each absence has cost in public cases, not a measured quantity, and a different reviewer would order the middle of the list differently. The precedents are also selection biased by construction: these are the failures that produced published reports, so controls whose absence fails quietly are underrepresented here rather than unimportant. Use the score to rank your own work, not to compare two companies, and treat the bottom of the list as under-weighted rather than safe.
The full argument is in your test suite is a diligence asset, and the discipline underneath it in a test that has never failed is a decoration. For the coverage number an acquirer will ask for, see coverage against mutation.
Questions
Concrete things rather than a document scan. Hardware focused diligence is an engineer led hands-on review over days: schematic review, bill of materials obsolescence risk, signal integrity, and firmware that will hold in production. Software checklists ask for coverage percentage, uptime and deployment frequency, flag vague answers as red flags, and separately warn that coverage numbers alone are gameable.
Because to a diligence engineer, a check nobody has watched fail is unproven, and unproven is indistinguishable from absent until they see it catch an injected defect. That middle state is the most common honest answer a team can give, and treating it as equivalent to having the control is exactly the error the whole exercise is about.
Ask them to make a check fail on camera. Pick a guard on a critical path, have an engineer break the thing it protects, and watch. It takes minutes and it converts a claim into an observation, which no amount of documentation does.
A verification data room with four artifacts: surviving mutant reports on critical paths, fault injection results, deploy consistency evidence, and a flaky test ledger. Diligence engineers now ask for exactly these, they are cheap to produce, and vague answers reprice deals.
Report it and expect it to be discounted. Once suite size is controlled for, coverage correlates only weakly with fault detection, and against real post release defects the correlation is statistically insignificant at Spearman rho of about -0.059. Coverage is a floor that flags an inadequate suite rather than evidence of a good one.