Withhold the Insight When It Does Not Reconcile

Mostafa Dhouib·11 min read

Withhold the Insight When It Does Not Reconcile

The short answer. A person reading a figure on a screen has no way to tell whether it ties back to the system it came from. Nothing about a wrong number looks wrong. So a well-written narrative over figures that do not reconcile is, to the reader, identical to a well-written narrative over figures that do, and the reader will act on both. That is why the correct output when reconciliation fails is nothing at all. Not a caveat, not a confidence badge, not a greyed-out figure with a warning icon. The reconciliation has to run as a gate before generation rather than as a report produced alongside it, and the product has to be designed for the case where it declines to answer.

Reconciliation usually arrives in a product as a feature. A page showing variances, a report an accountant can open, a nightly job that writes a log. Useful, and not what this is about. The claim here is narrower and more awkward. Reconciliation has to sit upstream of whatever the product says, as a condition on saying it, and the hard part of that is neither the checks nor the code.

The reader cannot tell

Start with the asymmetry, because every design decision follows from it.

An engineer looking at a figure that does not reconcile has tools. They can query the source, compare totals, trace the transformation. An operator looking at the same figure on a dashboard has one input, which is the figure, and one impression, which is that the product produced it.

Wrong numbers do not present as wrong. A cost percentage computed from an overstated sales figure sits in the plausible range. So does a labour ratio from an unburdened wage rate. A month-on-month movement driven by a delivery landing a day either side of a period boundary looks exactly like one driven by operational change. There is no anomaly to notice, which is what makes it dangerous rather than merely incorrect.

Now add generated narrative on top. The language model is fluent regardless of whether its inputs tie back to anything, and fluency is the one output property that is guaranteed while being uncorrelated with correctness. So the narrative supplies confidence the figures have not earned, and the reader reasonably treats that confidence as a signal.

The conclusion is uncomfortable. If the product cannot guarantee a figure reconciles, presenting it with a warning does not help either, because a warning competes against a confident paragraph and a clean layout, and it loses.

A gate, not a report

The structural difference is where the check sits.

As a report, reconciliation runs in parallel. The pipeline computes figures, generates insight, publishes the dashboard, and separately writes a variance report somebody may look at. The insight is published whether or not reconciliation passed, and someone is expected to cross-reference. Nobody does, least of all the operator the product was built for, because they are in a kitchen.

As a gate, reconciliation runs before anything is produced, in a fixed order. Ingest the source data. Reconcile each figure against the source system's own total for the same period. Only if every check passes, compute derived metrics. Only then, generate narrative. A failure short-circuits the pipeline and nothing downstream executes.

The checks themselves are not clever, and that is the point. Each compares a figure the pipeline computed against the same figure as produced by the system of record, which already knows the answer. Net sales against the point-of-sale system's own net sales report. Labour cost against what payroll recorded. Purchases against the supplier invoice total. Each has a right answer your code did not produce, which is what makes it a check rather than a self-consistency test.

Two design points decide whether the gate works.

The tolerance has to be explicit and tight. Zero is the right default for figures that are sums of the same underlying records. A tolerance exists to absorb known, named differences, such as a documented rounding convention, and each one has to be justified individually. A tolerance introduced to make the gate pass is the gate being removed, slowly, by somebody with good intentions.

The gate has to be per figure, not per pipeline. If labour reconciles and cost of goods does not, labour insight can still be produced. A single global pass or fail means one awkward feed takes down the whole product, which guarantees the tolerance creeps.

That granularity requires knowing which figures each output depends on. In practice each metric and each generated statement declares its inputs, and the gate suppresses anything whose dependency chain touches a failed check. It is bookkeeping rather than difficulty, and it gets skipped because at the start everything reconciles and the dependency map looks like overhead.

What to show instead

Silence does not mean a blank screen. It means not showing the insight, and showing something else that is true.

Name the failure in the operator's language. Not a stack trace and not a technical variance. Yesterday's sales do not match the till. Something along those lines, concrete, in the vocabulary of the business.

Show the gap. The two figures and the difference between them. An operator can frequently recognise the cause immediately, because they were there. A terminal that was offline, a cash-up that was done at the wrong time, a delivery entered twice.

Say what is affected and what is not. Labour figures are fine, food cost is unavailable for this period. That keeps the product useful on the days when one feed is broken, which are more common than the days when everything is broken.

Give one action. Usually something to check in the source system. The failure is almost always an operational event rather than a software defect, so the resolution sits with the operator rather than support.

Be explicit about recovery. Once the source data is corrected, the check passes and the insight appears. Stating that turns the gate from a wall into a step, which changes how it is received.

There is a side effect worth noticing. A product that surfaces these gaps becomes a data-quality tool, and in hospitality that is valuable in itself. The gate stops being the part that withholds and becomes the part that finds the till nobody closed out. That reframing is the most useful thing I know for getting it accepted.

The organisational problem is the real one

The engineering is a day's work. The difficulty is everywhere else, and pretending otherwise is why this pattern gets designed and then quietly disabled.

It demos badly. A product that always produces a confident paragraph demos better than one that sometimes says a figure is unavailable. Demos use clean data, so the gate never fires, which means the only time anybody sees it is when a prospect's real data breaks it in a sales conversation.

It looks like unreliability. To a buyer, a product that declines to answer looks less capable than one that answers. The distinction between declining and being wrong requires the buyer to believe that the competitor's confident answer might be wrong, and they have no way to verify that, which is the original asymmetry showing up in the sales process.

The pressure to add a tolerance is continuous and reasonable. Every instance is argued on its own merits by somebody sensible. This feed is always off by a small amount. This client's system rounds differently. We are blocking a customer over a trivial difference. Each argument is locally correct and the cumulative effect is that the gate no longer gates.

Nobody is rewarded for the silences. The value of the gate is the wrong decisions that did not happen, which are unobservable by construction. The cost is visible every time it fires. That asymmetry is the same one that gets reconciliation deferred in money-movement systems, and it has the same outcome if nobody owns the decision.

The defences I have seen work are organisational rather than technical. Put the gate in the acceptance criteria so it is a contractual property rather than an implementation choice. Make every tolerance require a named owner and a written justification, so adding one is a visible act. Report how often the gate fired and what it caught, so the silences have a number attached. And agree the principle before the first difficult case, because arguing it in the abstract is winnable and arguing it with a customer waiting is not.

The honest limit

A gate guarantees that a figure matches its source. It does not guarantee the figure is right. If the source is itself wrong, because a till was configured with the wrong tax treatment or a payroll rate was entered incorrectly, the check passes and the insight is confidently wrong. Reconciliation catches transformation and ingestion errors, which are the common ones, and is silent on errors upstream of the source.

It also says nothing about whether a derived metric is defined correctly. A labour percentage that reconciles perfectly to payroll while using an unburdened wage rate will tie and still mislead. Definitional errors sit outside the gate and need agreeing with somebody who knows the domain before anything is built.

There is a failure mode specific to the gate too. If it fires too often, people route around it, and a product unavailable more than it is available gets replaced by a spreadsheet. The gate is only viable on top of ingestion that is reliable most of the time. On a feed that genuinely cannot be made to tie, the answer is not a generous tolerance, it is not shipping that metric.

And this is a design position rather than a measured outcome. I have no figure for decisions avoided, because the thing the gate prevents leaves no trace. Judge it on the mechanism and the asymmetry it responds to rather than on a result I cannot evidence.

FAQ

Why is a warning on an unreconciled figure not enough? Because a warning competes with a confident paragraph and a clean layout, and it loses. The reader has no independent way to judge the figure, so the product's own presentation is their only signal. Showing the number at all supplies more confidence than the number has earned, and the reader will act on it. The only presentation that matches the uncertainty is not presenting the figure.

What is the difference between reconciliation as a gate and as a report? Position in the pipeline. A report runs alongside generation, so the insight is published whether or not the figures tie, and someone is expected to cross-reference. A gate runs before generation, so a failure short-circuits and nothing downstream executes. The checks can be identical. What changes is whether an unreconciled figure can reach a reader.

What does a reconciliation check actually compare? A figure the pipeline computed against the same figure as produced by the system of record for the same period. Net sales against the point-of-sale system's own net sales report, labour cost against payroll, purchases against the supplier invoice total. The comparison has a known right answer that your code did not produce, which is what distinguishes it from a self-consistency test.

What should the product display when a check fails? The failure named in the operator's language, the two figures and the gap between them, which figures are affected and which are still available, one action to take in the source system, and a statement that the insight returns once the data is corrected. Operators frequently recognise the cause on sight, because the usual causes are operational events they witnessed.

Why is this organisationally difficult to ship? Because it demos badly on clean data, because declining to answer reads as less capable than answering, because every request for a tolerance is individually reasonable, and because the value of the gate is the wrong decisions that did not happen, which nobody can count. The costs are visible each time it fires and the benefit is invisible always, which is the asymmetry that gets reconciliation deferred everywhere else too.

What does a reconciliation gate not protect against? Errors upstream of the source. If a till is configured with the wrong tax treatment or a payroll rate was entered incorrectly, the figure ties perfectly to a wrong source and the insight is confidently wrong. The gate also says nothing about whether a metric is defined correctly, so a labour percentage that reconciles to payroll while using an unburdened wage rate will pass and still mislead.