The starting point was a governance-grade specification: hundreds of requirement identifiers, recorded product decisions, and a large suite of blocking acceptance tests, all under version and hash control. The job was to turn it into an architecture somebody could build. That meant a scoped first release, a work breakdown traceable to function identifiers, and a procurement document another firm could bid from without having to read the governing documents themselves.
A record can be trusted when everything in it can be traced back to what was written first.
An append-only record is one where entries are only ever added. Nothing is edited and nothing is deleted. A correction is a new entry that refers to the old one and says what changed and why. This sounds wasteful, and it is the reason the design is used wherever history matters: the full sequence of what was known, and when, is preserved by construction, because there is no operation that could destroy it. Accounting ledgers, version control and audit trails all work this way.
Reading an append-only record directly is slow and awkward, so systems build derived layers on top of it: summaries, indexes, structured extracts, and a current state that shows how things stand now. A derived layer is defined by a rule that turns the source into the layer. The test of whether it really is derived is simple. Delete it, run the rule over the source, and see whether the same layer comes back. If it does, the layer is a convenience and can be trusted as far as the rule can. If it does not, the layer contains information that exists nowhere else, which makes it a source in its own right, usually one with no retention policy, no audit trail and no mention in the governing documents. Layers built by language models fail this test by nature, because running the same model on the same input can give a different result, so their outputs have to be stored as records along with the model version.
Regulated health software adds a second discipline: risk control verification. A risk control is a measure that reduces the likelihood or severity of a harm, such as a check that blocks an unsafe action or a warning shown at the right moment. Verifying it asks two separate questions. Was the control implemented as specified, which a test against the specification can answer. And does it actually reduce the risk it was designed for, which needs evidence about how it performs in use, for instance whether the warning is noticed and acted on. A control can pass the first and fail the second. Every requirement, control and test is tied together by identifiers so a reviewer can trace any of them to the others, and any change to validated behaviour, including a change nobody on the team initiated, goes through documented change control.
Three things in the specification shaped the architecture more than any feature did. The blocking tests needed a release of their own. The global safety gate had to exist as a component before the first release was scoped, because everything else is scoped against it. And the record had to be rebuildable: an append-only source, a governed derived structure, and a current state with history, where the derived layer can always be regenerated from the source. The moment a derived layer cannot be regenerated, it has quietly become a source that no governing document describes.
Governed systems usually fail at the seams with things nobody governs.
A specification under version and hash control is a strong defence against undocumented change inside the specification. The failures that get through tend to sit at its edges: a derived table that someone started editing by hand, a model behaving differently after a vendor upgrade, an inference request routed somewhere it was never meant to go. None of these changes a governed document, so none of them triggers the controls built around those documents.
The other pattern is verification done halfway. A team writes a test confirming that a risk control is present and behaves as specified, marks it verified, and moves on. Whether the control achieves its purpose in use is a second question with its own evidence, and plans that conflate the two tend to discover the gap at an audit.
The reflex, and the fix.
Scope the first release by feature value
That is how release planning normally works, and it produces an increment people can see and try.
The safety gate is global. Scoped around, it gets renegotiated in every release that follows, and the blocking tests have no release they belong to.
Give the safety gate its own release
A first release that delivers infrastructure and few visible features, and a rebuildability rule applied to every derived layer.
A gate that every later release is scoped against, and a record whose derived layers can be regenerated and checked against their source.
Build what everything else depends on first, even though it is the hardest thing to demonstrate.
Three layers and one rule between them
An append-only source, a derived structure governed by an explicit rule, and a revisable current state that keeps its history. The rule is that the derived layer must be rebuildable from the source. If it is not, it is a source, and it has to be governed as one.
Model outputs are stored, never replayed
A layer built by a model cannot be regenerated by running the model again, because the same inputs do not reliably produce the same outputs, even at the lowest temperature setting. The output and the version of the model that produced it are therefore kept as part of the record.
Model retirement goes through change control
Hosted models are retired on published schedules, and some deployment types upgrade automatically. For software whose behaviour has been validated, an upgrade arriving on a vendor's timetable is a regulatory event, so the architecture pins versions and treats any change of model as a controlled change.
Residency is established before it is relied on
Where inference physically runs is not always selectable, and cross-region routing can place it outside an intended boundary. That has to be established for every inference path before a design depends on it, and re-verified whenever the vendor's terms change.
Every risk control verified twice
The governing standard requires each risk control to be verified for implementation and for effectiveness. Those are different tests, and a plan that writes only one of them has done half the work.
What each choice cost, and what it bought.
AGiving the gate its own release
A first release that delivers a safety gate and the infrastructure under it is hard to show to stakeholders, because there is little to click on. That was the cost, and it was paid in patience and explanation. What it bought was a gate every later release is scoped against, and a home for the blocking tests. Scoping the first release by feature value would have produced something more visible and left the gate to be renegotiated in every release after it.
BApplying the rebuildability rule everywhere
Checking every derived layer for a rebuild path takes time, and some layers turn out to need reclassifying as sources with their own governance. That is extra design work and extra documentation. In return, no layer in the system can silently become a source, so the governing documents describe everything that actually holds information.
CStoring model outputs as part of the record
Recomputing a model-built layer on demand saves storage and keeps the record small. It also produces a different answer some of the time, and in a regulated record a value that changes on recomputation cannot be audited. Storing each output with the model version that produced it costs storage and adds the outputs to what must be retained and governed. It makes the layer reproducible by lookup, which is the only kind of reproducibility a model offers.
DWriting a procurement document that stands alone
The governing documents are long and precise, and asking a bidding firm to read them all would narrow the field and invite inconsistent readings. Translating them into a self-contained procurement document took real effort and introduced a risk of drift from the source. Tying every work item back to its function identifier controls that risk, and the result is a document that lets every bidder price the same, traceable understanding of the work.
A single headline number hides where a system fails. This work was scored on the dimensions that actually decide whether it holds in production, measured on real, held-out cases rather than the demo path.
Third-party platform and regulatory facts are cited with their source and date. No client outcomes are published.
A first release scoped by features, a safety gate renegotiated every time, and derived tables built by a model that nobody can regenerate once the model is retired.
A safety gate delivered as its own release, a three-layer record with an explicit rebuildability rule, model outputs and versions stored as part of the record, the residency of every inference path established, and risk controls verified for implementation and for effectiveness separately.
What it owns, and what it hands to a person.
Vendor retirement schedules, residency options and the regulatory position all move, and each was established at a point in time with its source recorded. The architecture depends on them holding, so a build that starts later has to re-verify each one before relying on it. Regulatory approval itself, and the clinical validity of the rules the system enforces, are outside this work.
Every derived data layer should be rebuildable from its source, and the ones built by a model usually are not. Store what the model produced and which version produced it, and the layer becomes a record that can be audited. Leave it as a cache and it becomes an undocumented source the first time the model changes.
An append-only source keeps history complete by construction, because corrections are added as new entries and nothing is ever overwritten.
Test every derived layer by asking whether it could be deleted and regenerated from its source. If it cannot, it is a source and needs governing as one.
Model outputs cannot be reproduced by running the model again. Store each output and the model version that produced it as part of the record.
In validated software, a vendor retiring or automatically upgrading a model is a change-control event. Pin versions and plan for retirement before it is announced.
A risk control needs two verifications: that it was implemented as specified, and that it actually reduces the risk it exists for.