Settlement And Reporting Are Projections Of One Record, Not Two Systems That Agree

Mostafa Dhouib·10 min read

Settlement And Reporting Are Projections Of One Record, Not Two Systems That Agree

The short answer. If settlement keeps its own copy of what happened and reporting keeps another, the two copies will diverge, and when they do nobody can say which one is correct, because both are originals. The fix is not better reconciliation. It is to make one append-only record of trade transitions the only history, and to derive both settlement and reporting from it as projections that hold no independent truth. Divergence then becomes structurally impossible rather than merely unlikely, because there is nothing for the two views to disagree about. What you take on instead is rebuild discipline, which is a smaller and much better understood problem.

A platform takes a quote through to an executed trade. Two things then need to happen to that trade. It has to settle, and it has to appear in reports that someone outside the business will read.

The obvious way to build that is two tables, or two services, each holding the state it cares about. Settlement tracks what is owed and paid. Reporting tracks what was traded, at what price, under which agreement. Both get written when a trade executes.

That is the decision that costs you, and it costs you months later.

Two copies means two originals

Here is the failure, concretely.

A trade executes. The settlement write succeeds. The reporting write fails, or succeeds against a slightly different version of the trade, or succeeds and is then amended by a later correction that only one of the two paths received.

Weeks pass. A report goes out. Someone compares the reported volume against what settled and finds a gap. Now the question is which number is right, and the honest answer is that you cannot tell, because both stores are originals. Each was written directly from an event that is no longer available to inspect. Neither can be checked against anything upstream, because there is nothing upstream.

What happens next is predictable. Somebody picks the number that looks more plausible, adjusts the other to match, and the discrepancy is closed without ever being explained. The platform has now been taught to resolve disagreements by preference, and the next one will be resolved the same way.

The deeper problem is that this is not a bug you can fix. Two independently maintained copies of the same facts will diverge given enough time, because divergence only needs one partial failure, one out-of-order update, or one code path that writes to one store and forgets the other. You can reduce the rate. You cannot remove the possibility, because the possibility is in the shape of the design.

One append-only record is the history

The alternative is to decide, up front, that there is exactly one place where what happened is recorded, and that place is append-only.

Model the trade as a state machine with explicit states such as quoted, accepted, executed, settled, and defined transitions between them. Every transition is written as an entry. The entry says which trade, which transition, with what values, when, and on whose authority. Entries are never updated and never deleted. The current state of a trade is not a column that gets overwritten. It is what you get by replaying the entries for that trade in order.

Three properties follow from append-only, and all three are load-bearing.

History survives. A row that is updated in place loses its previous value. An entry log keeps every version, because a change is a new entry rather than an overwrite. When someone asks how this trade reached these terms, the answer is readable rather than reconstructed.

There is a single order. Every consumer sees the same transitions in the same sequence. Two consumers reading one ordered log cannot form different beliefs about what happened, only about what to do with it.

Writes are naturally idempotent-friendly. Each entry carries the key of the transition that produced it, so a repeated attempt can be rejected at the write, and a repeated delivery downstream can be recognised and skipped.

Settlement and reporting become read models

Once the entry log is the history, settlement and reporting stop being systems that hold facts and become views over facts they do not own.

A projection is a derived store shaped for how it is read. Settlement needs obligations grouped by counterparty and date, with what is due and what has been paid. Reporting needs volume and value cut by instrument, period and agreement. Those are different shapes, which is exactly why they are separate projections. The mistake was never having two tables. It was having two sources of truth.

The rule that makes it work is one sentence: a projection may be rebuilt from the log at any time, and nothing is lost.

That rule has teeth. It means a projection can hold no fact that is not derivable from the log. The moment someone adds a column to the settlement projection that is edited directly, by a human or by a job, the projection has become an original again and you are back to two histories. The most common version of this is an operational flag, something like marked as reviewed or disputed, written straight onto the projection because it felt like metadata rather than a fact. If it matters, it is a transition, and it belongs in the log as an entry.

Projections lag, and that is fine if you say so. A read model updated asynchronously is briefly behind the log. The design decision is where that lag is acceptable and where it is not. Reporting can usually tolerate seconds. A pre-trade exposure check usually cannot, and should read the authoritative state rather than a projection. Deciding this per view is far better than pretending everything is instantaneous.

Projections can be wrong without the record being wrong. This is the quiet benefit. A bug in the reporting projection produces a wrong report and leaves the history intact. Fix the code, rebuild, and the report is correct. Under the two-originals design the same bug corrupts data you cannot recover, because the thing that was wrong was also the thing that remembered.

Rebuilding a projection without inventing a second history

Rebuild is the operation that makes projections safe, so it has to be a routine capability rather than an emergency procedure. The practical requirements:

Make rebuild part of normal operation. Run it on a schedule in a non-production environment and compare the output against the live projection. A rebuild that has never been run is a rebuild that does not work, and you will discover that at the worst moment. The comparison also doubles as a drift detector.

Build into a new store, then switch. Rebuilding in place leaves a window where the projection is partial and queries against it are wrong in a way that looks like data loss. Build the new version alongside, verify it, then move reads across. If something is wrong, you still have the old one.

Keep processing position explicit. Each projection records how far through the log it has consumed. That is what lets a rebuild resume, what lets you measure lag honestly, and what tells you whether a projection is stuck as opposed to merely quiet.

Handle replay of external effects separately. This is the trap. Rebuilding a read model is harmless. Replaying a log through a component that sends payment instructions or files reports is not. Projections must be pure, with side effects living in a clearly separate consumer that tracks what it has already emitted and refuses to emit twice. If the same code both derives state and sends instructions, you can never safely rebuild.

Version the projection, not the log. When the shape of a read model changes, give it a new version, build it from the log, and retire the old one once reads have moved. The log itself stays as it was written. Entries are facts about the past and the past does not get a schema migration.

What you gain is an answerable question

The practical payoff is not elegance. It is that a certain class of conversation ends.

When settlement and reporting disagree under the two-originals design, the question is which one is right, and there is no procedure that answers it. Under projections, the same disagreement means a projection is behind or has a bug, and both are diagnosable in minutes. Compare each projection's position against the log, replay the entries for the trade in question, and the mechanism is visible.

The same applies to the question an auditor or a counterparty asks, which is never "what is the balance" but "how did this become the balance". A log of transitions answers that directly. Two mutually adjusted copies answer it with an apology.

The honest limit

This is not free, and the costs are real.

You are now operating an append-only store that only grows, which means archival and partitioning become live concerns rather than someday concerns. You have introduced eventual consistency into read paths that were previously synchronous, and every screen now needs a stated answer about what staleness it tolerates. Debugging requires reading entries in order rather than looking at a row, and teams used to inspecting current state find that genuinely slower at first.

The pattern also does not help with the problem upstream of it. If the transitions being recorded are wrong, because an entitlement was misapplied or a price was computed incorrectly, the log faithfully records the wrong thing and every projection reproduces it consistently. Structural agreement is not correctness. It only guarantees that your views agree about a history you can inspect.

And it is possible to adopt the vocabulary without the discipline. A system with an entry log that also permits direct edits to projections has the costs of both designs and the benefits of neither. The single rule that keeps it honest is that a projection must be rebuildable from the log with nothing lost, and that rule has to be enforced in review, not just agreed in principle.

FAQ

Why can't I just reconcile settlement and reporting on a schedule? Because reconciliation finds disagreements and cannot resolve them. When both stores were written independently from an event that no longer exists, both are originals, and there is no upstream record to check either against. The usual outcome is that somebody picks the more plausible number and adjusts the other, closing the gap without explaining it.

What makes the append-only record different from a table with history columns? An entry log never overwrites, so every version of every value survives and the current state is derived by replaying entries in order. History columns record that something changed without preserving the sequence that produced the current value. The log also gives every consumer one shared order of events, which is what stops two views forming different beliefs.

Are settlement and reporting really allowed to be separate stores? Yes, and they should be. They are read for different questions and so want different shapes. The rule is that neither holds a fact that is not derivable from the log, so both can be rebuilt from it with nothing lost. Two tables are fine. Two sources of truth are not.

What happens when a projection has a bug? The projection is wrong and the history is not. You fix the derivation, rebuild the read model from the log, and the view is correct again. Under a design where the read store is also the record, the same bug corrupts data you cannot recover, because the thing that was wrong was also the thing that remembered.

How do I stop a rebuild from re-sending payments or refiling reports? Keep projections pure and put side effects in a separate consumer that records what it has already emitted and refuses to emit twice. Rebuilding a read model is harmless by construction. Replaying a log through code that both derives state and sends instructions is not, which is why the two must not share a component.

Where does an operational flag like disputed or reviewed belong? In the log, as its own entry. Writing it directly onto a projection turns that projection back into an original, because it now holds a fact that a rebuild would destroy. If a flag is worth keeping, it is a transition with an actor and a time, and the projection should derive it like everything else.