Ground Truth

The Delete-the-Variable Test

Mostafa DhouibMostafa Dhouib··7 min read
The short answer

A ten-second check that tells you whether a variable is doing real work or is scaffolding: delete it, rerun, and see what changes. Delete TCP's congestion window and the internet falls over, which it did in 1986. Delete your agent's iteration cap and the trace does not flinch. The test has one important limit, and knowing it is what stops the test becoming a licence to remove your own safety nets.

The Delete-the-Variable Test

The short answer. To find out whether a variable is load-bearing or decorative, delete it and rerun. If the system behaves the same, it was never doing anything. Delete TCP's congestion window and throughput collapses, which is not hypothetical: in 1986 congestion collapse dropped a link between two nearby buildings from 32 kbit/s to 40 bit/s. Delete your agent's iteration cap and it makes the same calls in the same order, just for longer. One is load-bearing. The other is scaffolding. The test takes ten seconds and it has one limit that matters enormously.

Systems accumulate variables that everyone assumes are doing something. Caps, thresholds, retry counts, timeouts, feature flags, defensive branches. They were added for a reason, the reason is undocumented, and nobody will remove them because nobody is sure.

There is a cheap way to find out.

The test

Delete it. Rerun. See what changes.

That is the whole method, and its value is that it converts a belief into an observation. You are not arguing about whether the variable matters. You are looking at whether removing it changes behaviour.

Delete TCP's congestion windowthe internet falls over1986: a link between two nearby buildings dropped from 32 kbit/s to 40 bit/s
Delete your agent's iteration capsame calls, same order, longerthe trace does not flinch
One is load-bearing. The other is scaffolding: real, correctly enforced, and bounding nothing you care about.
FigureThe ten-second test for whether a variable is doing anything: delete it, rerun, and see what changes.

Two worked examples make the difference vivid.

TCP's congestion window. Remove it and throughput does not degrade gracefully, it falls off a cliff. This is documented history rather than a thought experiment: in 1986, congestion collapse took a link between two buildings a few hundred yards apart from 32 kbit/s down to 40 bit/s. Roughly a thousandfold, on a network that was working the day before. The variable was holding the system up.

Your agent's iteration cap. Remove it and the agent makes the same tool calls, in the same order, producing the same intermediate results. It runs longer. The trace does not flinch.

Same test, opposite results, and the second one tells you that a variable everybody treats as the safety mechanism is not participating in the behaviour at all.

Why a variable can be provably correct and still be doing nothing

The uncomfortable part of the agent case is that the cap is not broken. It is a genuine, valid termination proof: the count decreases by one each pass and bottoms out at zero, so the loop provably stops.

Formal methods has a precise name for this. State added solely so a proof goes through, without changing what the program computes, is called ghost state.

The proof is real
Add a counter to an infinite loop
It is a function of state
It strictly decreases and is bounded below
The loop provably terminates. Publishable, correct
The program is unchanged
Same instructions, same order, same results
It now does them a thousand times and stops
Nothing the program computes is different
The proof answers a question nobody asked
Satisfies the review question without participating in the behaviour. That is the shape of every scaffolding variable.
FigureGhost state is the formal methods name for state added purely so a proof goes through. Both columns are true at the same time, which is what makes it hard to argue about.

Take an infinite loop. Add a counter that stops it after a thousand passes. You now have a real termination proof, publishable, correct. The program does exactly what it did before, a thousand times, and then stops. Both statements are true simultaneously: the proof is real, and the proof is worthless.

That is the shape of every scaffolding variable. It satisfies something, usually a review question or an audit item, without participating in the behaviour anyone cares about.

The limit that matters: Chesterton's fence

Here is where this technique becomes dangerous if you take only the headline, so it needs stating plainly.

A variable that guards a rare failure looks exactly like scaffolding until the rare failure happens.

Delete a retry limit and nothing changes, because the upstream service happened not to be flapping this week. Delete a bounds check and nothing changes, because no input was malformed today. Delete a circuit breaker and nothing changes, until the dependency degrades and your system takes the whole cluster down with it.

The test measures whether a variable is participating in the runs you observed. It does not measure whether it would participate in the runs you did not.

So the honest version of the method has a second question attached:

The gate
You deleted it and reran. Did the behaviour change?
Load-bearing. Put it back and stop
you have your answer
Now ask what it was supposed to catch
and whether that condition occurred in your test
If you can name the condition and it did not occur, this is a fence and you have not tested it. Construct the condition and rerun.
If you cannot name any condition under which it would fire, it is scaffolding.
A variable guarding a rare failure looks exactly like scaffolding until the rare failure happens.
FigureThe honest version of the delete test has two questions, and the second one is what stops it becoming a licence to remove your own safety nets.

Delete it and rerun. If behaviour changes, it is load-bearing. Stop; you have your answer.

If behaviour does not change, ask what it was supposed to catch. Then ask whether that condition occurred in your test. If you can name the condition and it did not occur, this is a fence and you have not tested it. Construct the condition deliberately and rerun. If you cannot name any condition under which it would fire, or the condition is one the system cannot reach, it is scaffolding.

The agent iteration cap survives this second question badly, which is what makes it a clean example. Name the condition under which it fires: the model keeps requesting tools past the limit. That happens constantly. It fires, and when it fires the run returns something indistinguishable from a completed answer. It is not a fence guarding a rare event; it participates all the time and the participation is harmful.

Where else this pays

The technique generalises past AI, and these are the places I reach for it most.

Timeouts nobody tuned. A thirty-second timeout on a call that always returns in under a second is not protecting you from anything you have observed. Ask what it catches. If the answer is a hung dependency, that is a fence, and you should test it by hanging the dependency rather than by deleting the timeout.

Retry counts. Three retries on an operation that has never failed twice is a guess. It is also, if the operation is not idempotent, three chances to double-execute. Here the test has teeth in both directions: deleting it may change nothing, and keeping it may be actively harmful.

Defensive branches. Code handling a case that cannot occur. Delete it and add an assertion that the case does not occur instead: now you learn if you were wrong, rather than silently handling something that never happens.

Config flags. Every long-lived system accumulates flags nobody flips. Deleting one is the cheapest way to discover it has been dead for two years, and dead flags are worse than useless because they make the configuration look like it has more surface than it does.

Monitoring and alerts. An alert that has never fired is either protecting you from something rare or watching something that cannot happen. The distinction matters and nobody checks, which is the same reason a test that has never failed proves nothing.

How to run it safely

The obvious caveat: do not do this in production.

Run it on a copy, with production-shaped traffic, and compare traces rather than outcomes. Comparing outcomes tells you whether the answer was the same. Comparing traces tells you whether the path was the same, which is the question, because a variable can change the path without changing the result until the day it does.

Where a variable genuinely cannot be removed for a test, the weaker version is to log every time it fires. A cap that never fires and a cap that fires on a third of runs are extremely different objects, and most systems cannot currently tell you which they have. That single counter is often the whole finding.

The one sentence

Ask of any variable: if I delete this, what changes, and if nothing changes, what was it supposed to catch and did that happen?

The first half finds scaffolding. The second half stops you tearing down a fence.

FAQ

How do I know whether a variable in my system is doing anything? Delete it, rerun on a copy with production-shaped traffic, and compare traces rather than outcomes. If behaviour is unchanged, it was not participating in the runs you observed. Then ask what it was supposed to catch and whether that condition occurred during your test.

What is ghost state? State added purely so a proof goes through, without changing what the program computes. Adding a counter to an infinite loop produces a genuine termination proof and an unchanged program, which is how a proof can be both real and worthless.

Is an agent's max-iteration cap doing anything? It bounds a single run, which is real. But deleting it produces the same tool calls in the same order, just for longer, so it is not participating in the behaviour. It also fires frequently, and when it does the run returns something the caller cannot distinguish from a completed answer.

Doesn't this encourage removing safety mechanisms? Only if you stop at the first half. A variable guarding a rare failure looks identical to scaffolding until the failure occurs, so the second question is mandatory: name the condition it catches and check whether that condition occurred in your test. If you can name it and it did not occur, construct it deliberately and rerun.

What if I cannot remove the variable to test it? Log every time it fires. A cap that never fires and one that fires on a third of runs are very different objects, and most systems cannot currently say which they have. That counter alone is frequently the whole finding.

Carrying a program like this one?

Tell us the system, the stakes, and the date that matters. You get a straight technical reply from the person who would lead the work, within 24 hours.

Bring us the program