Stop Chasing 100 Percent Mutation Score

Mostafa Dhouib·August 26, 2026·5 min read

Stop Chasing 100 Percent Mutation Score

The short answer. Equivalent mutants produce behaviourally identical programs, so nothing can kill them and 100 percent is literally unreachable. Prevalence across the literature runs 4 to 39 percent of all mutants. Track the surviving mutant list on critical paths, which is actionable, rather than a headline percentage, which is not.

There is a predictable failure mode when a team adopts mutation testing. They run it, get a number, and the number becomes the target. Within a quarter somebody is writing tests to kill mutants rather than to check behaviour, and the practice has been converted into the thing it was supposed to replace.

What an equivalent mutant is

A mutation that produces a program behaviourally identical to the original. Same output for every possible input.

No test can kill it, because there is no input on which it differs. It is not a gap in your suite. It is not a gap in anything. And it appears in the report as a survivor, indistinguishable from a real, closable test gap.

Deciding which survivors are equivalent is undecidable in general. That is worth stating carefully, because framed as a theory result it sounds academic. Framed practically it is: you cannot, in general, tell whether a surviving mutant is a real gap or an impossible-to-kill no-op. Somebody has to look, one at a time.

The prevalence is the number that ends the argument. Madeyski, Orzeszyna, Torkar and Jozala, in IEEE Transactions on Software Engineering in 2014, report equivalent mutant prevalence between 4 percent and 39 percent of all mutants across the literature.

A metric with an unknown denominator somewhere between 4 and 39 percent off is not a metric you can set a target on.

What Google does instead

This is the most useful part, because it comes from the largest organisation publishing on the practice, running it across a monorepo of roughly two billion lines of code.

Petrovic and Ivankovic at ICSE-SEIP 2018, and Petrovic, Ivankovic, Fraser and Just in IEEE TSE 2021.

They do not compute a classic mutation score at all. Because they generate a probabilistic subset and only one mutant per line, there is no meaningful denominator to divide by, and they did not go looking for one.

The goal is not a number. It is to surface productive mutants as actionable findings during code review, in line, where an engineer can act immediately.

And the metric they do optimise is the useful one: usefulness. They drove the unproductive, "not useful" mutant rate from roughly 80 percent down to roughly 15 percent using context based suppression of arid nodes, meaning code where mutations are uninteresting, such as logging and trivial getters. About 85 percent of surfaced mutants are now actionable.

That reframing is worth stealing wholesale. The question is not what fraction of mutants died. It is what fraction of the things you showed an engineer were worth their attention.

The cost side, honestly

Naive full repo mutation takes hours, because the entire test suite is executed once per mutant. That is why nobody runs it that way at scale, and why Google's approach exists at all.

Some mitigations cost nothing in detection: per test coverage analysis so only tests touching the mutated line run, mutating only changed files, caching, and higher concurrency.

Sampling does cost detection, and this is the trade to state out loud rather than discover. Sampling around 10 percent of mutants cuts cost substantially and studies report roughly a 16 percent loss of fault detection power at that level. Cheap is available; free is not.

One further finding justifies the cheapest structural choice. In more than 90 percent of lines that have multiple mutants, either all are killed or none are. So one mutant per line captures almost all of the signal at a fraction of the cost, which is why Google generates exactly that.

What to track instead

The surviving mutant list on critical paths. Money, safety, access control. Each survivor names an assertion you do not have, in a specific place. That is an object a person can act on, unlike a percentage that moved from 71 to 74.

A stop rule rather than a target. If more than zero mutations survive on a safety path, close them before shipping. That is a gate on a small, important set, and it does not degrade into metric gaming, because the set is small enough that everyone can see what is in it.

Usefulness, if you are running this at scale. What share of surfaced mutants did an engineer act on? If that number is low, suppress more aggressively before you conclude the practice does not work.

The evidence that killing injected faults tracks catching real ones is in do injected faults track real bugs, and it carries its own ceiling: 17 percent of real faults couple to no mutant even in principle. Between that ceiling and the equivalent mutant floor, the honest position is that mutation score is a useful signal with two known, quantified sources of error, which is exactly the sort of thing that should never become a target.

The general argument is in a test that has never failed is a decoration.

FAQ

What is an equivalent mutant? A mutation that produces a program behaving identically to the original on every possible input. No test can kill it, and it appears in the report as a survivor indistinguishable from a real test gap.

Why can I not reach 100 percent mutation score? Because equivalent mutants cannot be killed by anything, and their prevalence runs 4 to 39 percent of all mutants across the literature. Deciding which survivors are equivalent is undecidable in general.

Does Google chase a mutation score? No. They do not compute a classic score at all, because they generate a probabilistic subset with one mutant per line. They optimise the usefulness of surfaced mutants, having driven the unproductive rate from about 80 percent to about 15.

Should I sample mutants to save time? It works and it costs detection. Sampling around 10 percent reports roughly a 16 percent loss of fault detection power. Per test coverage analysis, incremental mutation and caching are cheaper without that cost.

What should we track instead of a score? The surviving mutant list on critical paths, with a stop rule that more than zero survivors on a safety path blocks shipping. Each survivor names a missing assertion in a specific place.