Heartbleed Was One Missing Negative Test
Mostafa Dhouib··6 min readOpenSSL's heartbeat handler copied as many bytes as the client declared without checking that against what the client actually sent, returning up to 64 KB of adjacent memory. The tests only ever sent well formed heartbeats, so the failure path was never driven once, and the fix was as trivial as the bug.
Heartbleed Was One Missing Negative Test
The short answer. The handler read a client supplied payload length and copied that many bytes without checking it against the data actually sent, returning up to 64 KB of adjacent process memory. The tests only ever exercised well formed heartbeats where declared equals actual, so the failure path was never driven.
Heartbleed is usually filed under memory safety, and that filing is what stops people learning anything from it. C made the consequence severe. It did not make the test absent.
What actually happened
OpenSSL's TLS heartbeat handler read a client supplied payload_length field and performed a memcpy of that many bytes from the request into the response. It did not check that payload_length matched the amount of data the client had actually sent.
So a client could declare a large length while sending a tiny payload. The server would copy the declared number of bytes, run off the end of the request buffer, and return up to 64 KB of adjacent process memory per request: private keys, session data, credentials, whatever happened to be nearby.
RFC 6520, section 4 already required that a HeartbeatMessage whose payload_length is larger than the message can support must be discarded silently. The requirement existed, in the specification the implementation was written against. The check was simply absent.
Disclosed 7 April 2014, as CVE-2014-0160, and fixed in 1.0.1g.
Why the tests passed
The existing tests exercised well formed heartbeats, where the declared length equals the actual payload length.
That suite was not weak for what it covered. Every input it supplied was handled correctly, and it would have caught a regression in the normal path. It simply never supplied the one input class that mattered, and the code path that leaks memory was therefore never executed by anything, ever.
This is worth separating from carelessness, because the mechanism is structural. An implementer writes the tests they can imagine, and the tests they can imagine come from the same mental model that produced the code. If your model of a heartbeat is a message where the length field describes the payload, you will write tests where the length field describes the payload. The gap in the tests is precisely congruent with the gap in the code, because they share an author and a model.
That is why example-based testing systematically misses adversarial input, and why "write more tests" is the wrong prescription. More tests from the same model produce more of the same coverage.
The three things that would have caught it
Any one of these, independently.
A single negative test. Send a heartbeat with declared length far greater than actual and assert that the server does not return more data than was sent. It reads back foreign memory on the first run and fails immediately. This is the negative control in its plainest form: supply the input the happy path never supplies.
A property-based test. State the property rather than an example: the response never contains more bytes than the request supplied, or declared length never exceeds actual length. A framework then generates hundreds of inputs trying to break it and shrinks any failure to a minimal case. Properties catch the class, where examples catch the instance. Tools for this are ordinary now: Hypothesis, proptest, fast-check, jqwik.
A fuzzer with a sanitizer. A coverage-guided fuzzer mutates inputs to reach paths nobody wrote a test for, and AddressSanitizer instruments the binary so the out-of-bounds read fails loudly instead of quietly succeeding. This is the combination that matters most here, and it is worth understanding why.
Why a functional test would not have noticed anyway
Suppose the suite had happened to send a malformed heartbeat. Without instrumentation, it might still have passed.
An out-of-bounds read frequently succeeds. It returns whatever bytes are adjacent in memory, the function proceeds normally, and the response is well formed. An assertion checking that the server responded, or that the response parses, or that no exception was raised, all pass. The failure is not a crash. It is a well formed correct-looking answer containing somebody else's private key.
That is the general shape of the most expensive verification failures, and it recurs across every incident in this cluster. Units errors return plausible numbers. A wildcarded field returns a successful deployment. Detection here has to come from instrumentation rather than from assertion, because there is nothing in the output for an assertion to object to. A sanitizer build changes what your existing tests are capable of detecting, which is why it is worth running even before you write a single new test.
The rule this produces
Every parser, protocol and sensor input path gets a fuzz test and a negative test.
Those three categories share a property: the input arrives from outside your model. A parser accepts what somebody else emits, a protocol handler accepts what a peer sends, a sensor path accepts what physics produces, and in the interesting cases what arrives is adversarial rather than merely unusual. Your own imagination is the wrong generator for that input, which is the entire reason to hand the job to a fuzzer or a property.
The fix, in the words of the people who shipped it, was as trivial as the bug: just a matter of adding a value check. That asymmetry is the whole argument. The cost of the control was minutes. The cost of its absence was a global key rotation.
The general version of this argument is in a test that has never failed is a decoration.
FAQ
What caused Heartbleed? OpenSSL's heartbeat handler copied as many bytes as the client declared in the payload_length field without checking that against the data actually sent, returning up to 64 KB of adjacent process memory per request. RFC 6520 section 4 already required such messages to be discarded silently.
Why did OpenSSL's tests not catch Heartbleed? The suite only exercised well formed heartbeats, where declared length equals actual length, so the code path that overreads was never executed. The gap in the tests matched the gap in the code because both came from the same model of what a heartbeat is.
What kind of test would have found Heartbleed? Any of three: one negative test sending declared much greater than actual, a property-based test over the relationship between declared and actual length, or a coverage-guided fuzzer running against a sanitizer build.
Would a normal functional test have caught it if it sent a malformed heartbeat? Not reliably. An out-of-bounds read usually succeeds and returns a well formed response, so assertions on the response can all pass. Detection needs instrumentation such as AddressSanitizer rather than an assertion on output.
How large was the fix? As trivial as the bug: adding a value check, restoring the bounds test the RFC already mandated.
Nine controls, each tied to a public failure that cost somebody nine figures or a life, with the four artifacts a verification data room holds and the one question an investor should ask in the room. Built to be printed and ticked only where somebody has actually watched the check go red.
Tell us the system, the stakes, and the date that matters. You get a straight technical reply from the person who would lead the work, within 24 hours.
Bring us the program