Exactly-Once in Effect Is Not Exactly-Once, and the Difference Is Where Your Ledger Stops

Mostafa Dhouib·6 min read

Exactly-Once in Effect Is Not Exactly-Once, and the Difference Is Where Your Ledger Stops

The short answer. You cannot get exactly-once delivery across a crash boundary, which is why every serious event transport offers at-least-once instead. What you can build on top is exactly-once in effect: deduplicate on the sender's event identifier, append accepted events to a ledger that is never edited, and derive state from the ledger rather than mutating it on arrival. That is the strongest available construction and it is not the same claim. It holds exactly as far as the identifier is a sound deduplication key, and there are specific cases where it is not. The honest statement names the dependency, and the backstop for everything outside it is reconciliation rather than the handler.

Teams that have been bitten once by duplicate processing arrive at the same architecture, and it is the right one. Key on the event identifier, append, derive.

Then the architecture gets described as exactly-once, and the gap between that phrase and what was built is where the next incident lives.

Why delivery cannot be exactly-once

The sender dispatches an event and waits for acknowledgement. The receiver processes it and the acknowledgement is lost.

The sender now has two options and no information. Resend, risking a duplicate. Or do not, risking a loss. There is no third option, because the thing the sender needs to know, whether the receiver processed it, is exactly the thing the lost message would have told it.

At-least-once chooses duplication. At-most-once chooses loss. Exactly-once would require the receiver's processing and the acknowledgement to commit atomically across a network boundary, which is the problem you were trying to solve.

So the transport cannot provide it. The question is what you build above it.

What the construction actually gives you

Deduplicate on the sender's identifier. Record which identifiers you have processed, and make a repeat a no-op. This is what absorbs duplication.

Append to a ledger, never edit. Each accepted event is written once and never changed, which makes the history reconstructable rather than inferred from a current value.

Derive state from the ledger. Compute the current position from the events rather than incrementing a counter on arrival. This is what absorbs reordering, because a late event is incorporated into the history correctly rather than applied over something newer.

Together these give behaviour indistinguishable from exactly-once for everything the transport does: duplicates, delays, and reordering within the retry window.

Where it stops

Three boundaries, and all three are worth naming in the design rather than discovering later.

A replayed event with a new identifier. If the sender reissues the same logical event under a fresh identifier, your handler sees something it has never seen, because that is literally true. Deduplication cannot help, since the key is new. This happens after a sender-side incident, a backfill, a migration between environments, or a manual resend by somebody trying to be helpful.

Events that were never delivered and are no longer available. The ledger is correct about everything it received and knows nothing about what it did not. Many platforms retry for a bounded window and offer no replay afterwards, so an outage longer than that window leaves a gap with no mechanism to fill it.

Two senders describing the same real-world fact. If the same underlying event arrives through two different channels with two different identifiers, the handler is correct to treat them as distinct and the business is not.

All three are outside what identifier-based deduplication can address, and none is exotic.

The backstop is reconciliation

For everything outside the handler's reach, the mechanism is a standing comparison against the source of truth.

The handler guarantees correctness for the transport's own failure modes. Reconciliation catches what is left: the replayed event with a new identifier, the gap from an outage, the duplicate logical fact from two channels. It is slower and it is visible, which is the correct trade against fast and wrong.

That also settles the authority question. The sender knows what happened. Your ledger is a projection that should agree with it, and when they disagree the sender is right and your ledger has a gap to investigate.

Say it precisely

The difference between these two sentences is the difference between a system you can reason about and one you cannot.

"The handler is exactly-once."

"The handler is exactly-once in effect over an at-least-once unordered stream, keyed on the sender's event identifier, with reconciliation as the backstop for replays under new identifiers and for gaps outside the retry window."

The second is longer and it tells a reader where the guarantee stops, which is the thing they need when something goes wrong at three in the morning. Documentation that overstates a guarantee is worse than documentation that omits it, because it stops people looking in the right place.

The honest limit

Even with reconciliation, some gaps are findable and not fillable. If a platform has no replay and the retry window has passed, knowing an event is missing does not recover its content. What reconciliation gives you is a known gap rather than an unknown one, and a known gap can be handled by asking a person, querying the source directly, or accepting it explicitly.

It is also worth saying that the ledger has a cost. Deriving state is more work than reading a column, which matters at volume and is usually addressed with a periodic snapshot plus the events since. That snapshot is an optimisation of the derivation. Treating it as the record reintroduces every problem the ledger removed, quietly.

FAQ

Why can't event delivery be exactly-once? Because if the acknowledgement is lost, the sender cannot tell whether the receiver processed the event, and the lost message is exactly what would have told it. Resending risks a duplicate and not resending risks a loss. Exactly-once would require processing and acknowledgement to commit atomically across the network, which is the original problem.

What does deduplicating on an event identifier actually guarantee? Exactly-once behaviour for the transport's own failure modes: duplicates, delays and reordering within the retry window. It guarantees nothing about events replayed under a new identifier, events never delivered and no longer available, or the same real-world fact arriving through two channels.

What happens if the sender replays an event with a new identifier? Your handler processes it, correctly, because it has genuinely never seen that key. This occurs after a sender-side incident, a backfill, an environment migration or a manual resend, and deduplication cannot address it. Reconciliation against the source is the backstop.

Why derive state from a ledger instead of updating a counter? Because updating on arrival is only correct if events come exactly once and in order, and they do neither. Deriving from an append-only history means a late or out-of-order event is incorporated correctly rather than applying a stale change over a newer one.

How should I describe this guarantee in documentation? Precisely enough that a reader knows where it stops. Exactly-once in effect over an at-least-once unordered stream, keyed on the sender's identifier, with reconciliation covering replays under new identifiers and gaps outside the retry window. Overstating it stops people looking in the right place during an incident.

Is a materialised snapshot of derived state safe? As an optimisation of the derivation, yes. As the record, no. Once the snapshot is treated as authoritative rather than as a cached computation, every problem the ledger removed comes back, and it comes back quietly because the snapshot still returns a plausible number.