Your Decimal Engine Is Still Eating Floats: Determinism Is Decided at the Parser Boundary

Mostafa Dhouib·8 min read

Your Decimal Engine Is Still Eating Floats: Determinism Is Decided at the Parser Boundary

The short answer. Moving a calculation engine to decimal arithmetic is the correct decision and it is usually applied one layer too late. A spreadsheet stores numeric cells as binary doubles. The standard dataframe library infers a float type when it reads them. And constructing a decimal from a float inherits the float's error exactly rather than correcting it. By the time your engine sees the value it has already been through two lossy steps, so the arithmetic is exact and the input is not. Determinism is a property of the parser boundary, not of the engine, and the fix is to read numeric cells as text and build the decimal from the string.

Somebody discovers that their financial or regulatory calculations are not reproducible. Small differences, last-digit differences, the kind that make a total fail to tie by a cent.

The diagnosis is correct and immediate: floating point. The fix is correct too: use a decimal type. The team switches the engine over, the change is substantial, everyone is satisfied, and the problem does not go away.

It does not go away because the conversion happened after the damage.

Three steps, two of them lossy

Trace a number from a spreadsheet cell to your engine.

The cell. A spreadsheet stores numeric cells as IEEE-754 doubles. The value you see as 2.15 is held as something fractionally different, because 2.15 has no exact binary representation. That is already true before any of your code runs, and it is unavoidable: it is how the file format works.

The read. The common dataframe library infers types when it loads. A column of numbers becomes float64. Even if the underlying library handed over a string, the inference converts it. This is a default, it is usually what people want, and it is silent.

The conversion. Then somebody constructs a decimal from that value. This is the step that feels like the fix and is not, because constructing a decimal from a float faithfully represents the float, including its error. You do not get 2.15. You get an exact decimal representation of the double that was standing in for 2.15, carried to a great many digits.

So the pipeline is: a lossy storage format, a conversion to float, and an exact-but-wrong conversion to decimal. Three steps, and your careful engine is the fourth.

Why nothing warns you

Every one of those steps is behaving correctly by its own contract.

The spreadsheet is storing a number the way its format stores numbers. The dataframe library is inferring a sensible type from numeric-looking data. The decimal constructor is doing exactly what it says: representing the value it was handed, precisely.

There is no error condition anywhere, because nothing went wrong in the sense any of those components understands. The loss is distributed across a sequence of individually correct operations, which is the hardest kind of defect to see.

And the symptom is the quietest available. The numbers are nearly right. A total is off by a cent. A threshold comparison flips in one direction on one row. A report produced today does not quite match the one produced last month from the same inputs.

The fix is at the boundary

Read numeric cells as text, and construct the decimal from the string.

Constructing a decimal from a string is exact. The string says 2.15, the decimal is 2.15, and nothing in between has had an opinion about it. This is the entire difference, and it is one argument in one function call at the point of ingestion.

That means the ingestion step has to be deliberate rather than default. You are overriding the library's helpful inference, specifying that these columns come in as text, and converting them yourself. It is less convenient and it is the only version that is correct.

Three things follow from treating ingestion as a contract rather than a convenience.

Fail loudly on anything unparseable. If a cell cannot be converted, that is a finding. A silent coercion to a null or a zero is how a missing value becomes a confident number further down.

Normalise locale at the boundary. A decimal comma is not a thousands separator, and which one it is depends on where the file came from. Deciding that at ingestion, once, is a great deal safer than discovering it in a total.

Write a canonical record. The output of ingestion is a clean, typed, exact representation that everything downstream reads. The messy formats stop at the boundary and never propagate.

The related trap: references without dates

A second thing that breaks reproducibility, and it is not about types at all.

Calculations depend on reference values: thresholds, rates, limits, classification bands. Those change. A regulatory framework is replaced and the numbers move.

If the reference store holds only current values, then a report produced two years ago cannot be reproduced today, because the engine now computes against different references. The arithmetic is deterministic and the result still differs, which is a confusing way to fail.

Reference values need effective dates, and the engine needs to resolve them as of the date of the thing being calculated rather than as of now. This is one of those decisions that costs very little at the start and cannot be retrofitted, because the historical values were never recorded.

Worth knowing if you are reaching for database features here: current PostgreSQL has added native valid-time temporal keys, but system-versioned tables and querying a table as of a past time are still not in it. The effective-dating is something you model rather than something you switch on.

What the deliverable actually is

On work like this the engine is the easy half. The hard half, and usually the real deliverable, is proof that each result matches the specification clause by clause.

That suggests a particular shape. Map each clause to one function, so each can be proven individually rather than as part of a combined output. Wire the specification's own worked examples in as a test suite, failing at first. Then produce a reconciliation table showing, for every clause, the specification's expected value against what the engine returns.

Doing that has a useful side effect. You find out whether the specification and its worked examples actually agree with each other, and whether any clause has no worked example at all. A clause with no example is a clause nobody can prove, and finding those is worth as much as implementing the rest.

The honest limit

Reading as text and constructing decimals from strings removes float contamination from ingestion. It does not make the engine correct, it does not validate the specification, and it says nothing about whether the number in the cell was right when somebody typed it.

It also costs something. Text ingestion with explicit conversion is slower than letting the library infer, and on very large files that is a real consideration. The answer is usually to apply it to the columns that feed calculations rather than to everything, which keeps the exactness where decisions are made.

FAQ

Does switching to a decimal type fix floating-point errors in my pipeline? Only if the values reach the decimal type without passing through a float first. A spreadsheet stores numeric cells as doubles, the usual dataframe library infers a float on read, and constructing a decimal from a float represents that float exactly, error included. The engine then computes precisely on an already-wrong input.

What is the difference between constructing a Decimal from a float and from a string? From a float, you get an exact representation of the binary double, which is not the number you wrote. From a string, you get the number you wrote. That single difference is the whole fix, and it lives at ingestion rather than in the engine.

How do I read spreadsheet numbers without losing precision? Read the numeric columns as text, overriding type inference, and construct the decimal from the string yourself. Then fail loudly on anything that will not parse rather than coercing it, normalise decimal separators at the boundary, and write a canonical typed record that everything downstream reads.

Why does nothing error when a pipeline loses precision this way? Because every step is correct by its own contract. The file format stores doubles, the library infers a sensible type, and the decimal constructor faithfully represents what it was handed. The loss is spread across individually correct operations, so there is no single point at which something went wrong.

Why can't I reproduce a report from two years ago even though the arithmetic is deterministic? Most likely your reference values have no effective dates, so the engine computes against today's thresholds rather than the ones in force when the report was produced. Reference data needs to be temporal and resolved as of the date being calculated, and that cannot be retrofitted because the historical values were never stored.

What proves a calculation engine is correct? A reconciliation table mapping each specification clause to one function and comparing the engine's output against the specification's own worked example, clause by clause. Building it also reveals whether the specification agrees with its examples and which clauses have no example at all, and a clause with no example cannot be proven by anyone.