Environmental EngineeringField engagement

Determinism is decided at the parser boundary, several steps before the calculation engine anyone is worried about.

Architecture and delivery lead.
The approach, in four words
exact from the cellone clause, one functionreferences are effective-datedthe proof is the deliverable
01Cell as textnever as a number
›
02Decimal from stringexact
›
03One clause, one functionprovable
›
04Dated referencesold reports reproduce
›
05Reconcileclause by clause
a decimal constructed from a float carries the float's error exactly, and nothing downstream will say so
What was at stake

An environmental engineering practice receives laboratory results as spreadsheets, in around nine different layouts. Calculation engines turn those results into regulated environmental metrics, and the metrics go into reports that a named engineer signs. A signed report with a wrong figure in it means a professional liability, a resubmission, and an awkward question about every report signed before it.

How this works

A decimal is only as exact as the thing it was built from.

Computers usually store numbers with a fractional part in binary floating point. The number is held as a binary fraction scaled by a power of two. That is fast and compact, and it has an awkward property: most decimal fractions cannot be written exactly in binary, in the same way that a third cannot be written exactly in decimal. A value as ordinary as a tenth is stored as the nearest binary fraction available, which is very slightly off. Display software rounds it back for the screen, so a person looking at a spreadsheet sees exactly what they typed while the file holds something a hair away from it.

A decimal type stores digits in base ten, so a tenth really is a tenth, and arithmetic on decimals follows the rules a person would apply on paper, including rounding exactly where a specification says to round. That is why regulated and financial calculations use it. The catch is construction. A decimal built from a string of digits holds those digits. A decimal built from a float holds the float's exact binary value, written out in full, error included. The decimal type is behaving correctly when it does this. It faithfully preserves what it was given, and what it was given was already off.

Spreadsheets store numeric cells as floats, and the common tools for reading them infer a float type as soon as a column looks numeric. So the most natural path from file to engine converts every value to a float before any code the engineer wrote has run. Asking the reader for text instead produces the short decimal string the stored value stands for, which for a figure a laboratory reported is the figure they reported, and a decimal built from that string is exact. Determinism then has a second half. The same inputs must meet the same rules and the same reference values every time, including when a report is regenerated in a later year. Effective dating exists to guarantee that second half.

Decimal built from the float
Cell read as a number
the stored binary value, slightly off
Converted to decimal
the binary error is copied in full
Exact arithmetic
on a value that was already wrong
Decimal built from the text
Cell read as text
the digits the laboratory reported
Decimal built from the string
holds exactly those digits
Exact arithmetic
on the value that was measured
Same cell, same engine, different reader
Choosing the decimal type is necessary. Choosing how it is constructed is what makes it work.
FigureThe same spreadsheet cell taken down two paths into the same decimal engine. The engine is identical in both. Only the reader differs, and the reader decides whether the result is exact or almost exact.
The constraint

The engagement already had the right rule: every number comes from a deterministic engine, and models only write the prose around it. The difficulty is that determinism is mostly lost before the engine runs. A spreadsheet stores a numeric cell as a binary floating-point value. The usual dataframe library sees a numeric column and infers a float type as it reads the file. Constructing a decimal from that float copies the float's error exactly. A decimal engine fed from the obvious path therefore computes precisely on numbers that were already slightly wrong, and nothing inside it can tell. The deliverable is proof, clause by clause, that each result matches the specification, and that proof only means something if the numbers are exact from the cell onward.

Where it usually breaks

Precision is usually lost on the way in, where nobody is checking.

Most review effort on a calculation system goes into the calculations, because that is where the specification is and where the visible logic lives. The arithmetic of a carefully written engine is rarely the problem. The trouble is the series of quiet conversions between the source file and the first line of engine code. Each one is a default in some library, and none of them raises an error.

The second pattern is temporal. A system built to produce this year's report correctly stores this year's reference values. Next year the values change, the store is updated in place, and every earlier report silently loses the ability to be reproduced. Nobody notices until someone asks for one to be regenerated, usually because it is being questioned.

01

The inferred type

A reading library sees a column of numbers and decides it is a float column. The conversion happens before any project code runs, and because it is a default, nothing records it.

02

The faithful decimal

A decimal is constructed from that float and carries its binary error in full. Every downstream calculation is exact, and exactly wrong.

03

The cell that is not a number

Laboratory sheets contain entries such as a below-detection marker or a note typed into a numeric column. A lenient reader turns them into blanks or zeros, and the report computes on a value nobody measured.

04

The overwritten reference

Reference values are updated in place. This year's report is right, and no earlier report can be regenerated against the values it was signed under.

05

The clause nobody can prove

A specification clause has no worked example. The engine implements someone's reading of it, and that reading is never checked against anything.

Laboratory spreadsheet
a numeric cell is stored as a binary float
Reader infers a float
a library default, nothing is logged
Decimal built from the float
the binary error is copied in full
Exact decimal engine
computes precisely on the wrong value
Signed report
almost right, and hard to reproduce
The failure point
A signed figure that is almost right
exact arithmetic on a value that was wrong on arrival
No step raises an error
The engine is the only step anyone reviewed
The fix belongs at the reader, which is the step nobody thinks of as code.
FigureHow a correctly written decimal engine ends up behind a slightly wrong signed figure. Every step behaves as designed, and the error enters at the reader, before any project code runs. Review effort spent on the engine finds nothing, because the engine is not where the error is.
The fork

The reflex, and the fix.

Road not taken

Build the engines and switch them to decimal arithmetic

Pull

Decimal is the correct type for regulated figures. The change is easy to make, easy to describe, and it looks like the whole fix.

Why not

By the time the engine runs, the damage is done. A decimal built from a float carries the float's error to the last digit, so the engine is exact about a value that was wrong on arrival.

Road taken

Pin the ingestion contract first

Accepted

Numeric cells read as text, a canonical record that fails loudly on anything it cannot parse, and reference values stored with the dates they were in force.

Bought

Numbers that stay exact from the laboratory's cell to the signed report, and a reconciliation table that proves each clause against its worked example.

Decision

Every guarantee downstream is inherited from how the number was first read, so the boundary is fixed before the engine is built.

How it was built
01Ingest as text
›
02Canonical record
›
03Engine per clause
›
04Dated references
›
05Reconcile
01

Read numeric cells as text

The file holds a double. Read it as a number and convert it later, and the double's error survives every conversion that follows. Read the cell as text and build the decimal from that string, and the value is the one the laboratory reported.

02

One clause, one function

Each clause of the specification maps to exactly one function. That makes the proof granular. A reconciliation table can set each function's output beside the specification's own worked example, line by line, and any disagreement points at a single clause.

03

Reference values carry the dates they applied

The regulatory reference framework changes from time to time. A report signed in an earlier year has to reproduce against the references in force when it was signed, so every reference value is stored with the period it applied to, and each calculation asks for the value in force on its own date.

04

Audit the specification before implementing it

Before any code, check that the specification and its worked examples agree with each other, and list every clause that has no example at all. A clause with no example is a clause nobody can prove, and the client needs to know which ones those are before the reports go out.

Inside the key decisions

What each choice cost, and what it bought.

AFixing the reader before writing the engine

The engagement was framed around the engines, and the obvious plan was to build them first and tighten the inputs later. Doing it the other way round delayed the visible work. It also meant every engine was written against a canonical record whose values were known to be exact, so the reconciliation that followed tested the engine's logic and nothing else. In the obvious order, a mismatch in the reconciliation could have come from the engine, the reader or both, and separating them would have cost more than the early start saved.

BFailing loudly on anything unparseable

A lenient reader that turns an odd cell into a blank keeps the pipeline running and quietly changes the result. A strict one stops, names the file and the cell, and makes somebody look. That costs interruptions, particularly while the set of known layouts is being learned. It buys the property that every number in a signed report came from a cell the system understood, which is the property the person signing actually needs. A layout outside the known set is flagged, never guessed at.

COne function per clause

Grouping related clauses into shared functions produces less code. It also makes the proof coarse, because a reconciliation mismatch then points at a function that implements several rules at once. Mapping each clause to its own function costs some repetition. It buys a reconciliation table where each row is a single clause, a single function and a single worked example, which an engineer or an auditor can check without reading any code.

DDating the references from the start

Storing only current reference values is simpler, and it works perfectly until the framework changes. Storing each value with the period it applied to costs a slightly more involved lookup on every calculation. It is also the only design that lets a prior-year report be regenerated exactly, and that capability cannot be bolted on later, because the history it needs would already be gone.

Read the cell as text
the reported digits, never a float
Parse into a canonical record
anything unparseable stops the run
Engine function per clause
one rule, one function, one proof
Look up dated references
the values in force on the report's date
Reconcile against worked examples
one row per clause
Each step only has to be right about its own job, because the step before it already was.
FigureThe order the engagement was built in. Exactness is established at the first step and every later step inherits it. The engine sits in the middle and is the least risky part of the chain.
How it was measured

A single headline number hides where a system fails. This work was scored on the dimensions that actually decide whether it holds in production, measured on real, held-out cases rather than the demo path.

Exactness from cell to reportAgreement with each worked exampleReproducibility of a prior-year reportClauses with no worked example
The gate
Does this clause have a worked example in the specification?
Verified by reconciliation
the engine's output matches the example line by line
Flagged as unproven
listed in the deliverable for the engineer to judge
The list of flags is part of the deliverable, not a gap in it.
FigureHow each clause is accounted for. No measured outcome figures are published for this engagement, so the result is described by what the reconciliation can and cannot prove. A clause with no worked example is flagged and handed back, never passed silently.
figures

No measured figures are published for this engagement. Outcomes are described qualitatively.

What it produces
Without this discipline

A decimal engine computing precisely on values that became floats on the way in. The figures are almost right, which is the hardest kind of wrong to notice, and an earlier year's report cannot be reproduced once the reference framework has moved.

This system

An ingestion contract that stops on anything it cannot read, exact decimal arithmetic from the cell onward, an engine function for each specification clause, references stored with their effective dates, and a reconciliation table that proves each clause against its worked example.

exact from the cellone clause, one functionreferences are effective-datedthe proof is the deliverable
The operating envelope

What it owns, and what it hands to a person.

Handled with confidence
Spreadsheet layouts in the known set
Clauses with worked examples
Prior-year reproduction within the dated reference range
Flagged for review
A layout outside the known set
A clause with no worked example
Out of scope by design
Judgements the specification leaves to the engineer
The honest limit

The reconciliation proves the engine against the worked examples the specification supplies, and no further. A clause without a worked example is flagged as unproven, and the list of flags is part of the deliverable. Reproducing a prior-year report also depends on the reference store having been dated from the beginning. Reference values that were overwritten before dating began cannot be recovered retrospectively, and reports from that period cannot be reproduced exactly. Judgements the specification leaves to the engineer stay with the engineer.

What it generalizes to

Any pipeline whose output someone signs has the same order of work. Make the data exact at the point where it enters. Give each rule its own function so it can be proven on its own. Date everything the rules depend on, so an old output can still be reproduced years later. The arithmetic engine is usually the safest part of the system.

What to take from this

A decimal built from a float carries the float's error exactly. Construct decimals from text, and check what type your file reader infers before trusting anything downstream of it.

Determinism is decided where data enters a pipeline. An exact engine cannot repair a value that was converted on the way in.

A reader that fails loudly on an unfamiliar cell is worth its interruptions, because a lenient one changes results without telling anyone.

Mapping each rule to its own function makes proof granular, so a disagreement with a worked example points at a single rule.

Store reference values with the dates they applied. Without that history no earlier output can be reproduced, and the history cannot be rebuilt afterwards.

How we engage

You have a system like this one.
Tell us where it stands.

Whether it is failing, not yet built, or about to meet a scale it has never seen, we can tell you what we see.

Start a conversation
mostafa@opulion.dev · Response within 24 hours · By inquiry