Change the Model, Restart the Submission

Mostafa Dhouib·September 7, 2026·7 min read

Change the Model, Restart the Submission

The short answer. For a cleared device, a modification that could significantly affect safety or effectiveness can require a new submission before it ships. Retraining a model is capable of being exactly that. So the ordinary machine learning response to drift, retrain and redeploy, has a cost measured in clearance cycles rather than in GPU hours. The engineering consequence is that you should architect to reduce how often retraining is necessary, and that is decided before the first submission rather than after it.

Two teams face the same drift. Their inputs have shifted, accuracy is down, and the fix is a retrain on more representative data.

The web team ships it on Tuesday. The device team is looking at a regulatory question, and the difference is not paperwork overhead. It changes what a good architecture looks like.

Why this asymmetry exists

A cleared device is cleared as a specific thing. The clearance rests on evidence about that thing's performance. Change it in a way that could significantly affect safety or effectiveness and the evidence no longer covers what you are shipping, so the change can require review before it goes out.

A model is software whose behaviour is determined by its training data. Retrain it on different data and its behaviour changes, including on inputs nobody tested. That is close to the definition of a modification that could affect performance, which is why the ordinary machine learning maintenance loop collides with the regulatory frame rather than fitting inside it.

There is a pathway designed for this. A Predetermined Change Control Plan lets a manufacturer specify in advance what modifications will be made, how they will be validated, and within what bounds, so changes inside that envelope can be made without a new submission each time. It is the right tool, and it has a property most teams miss: it is written before you need it, and it only covers what you thought to describe.

That is the whole engineering lesson. The flexibility you will have in year three is determined by what you wrote down in year one.

What this should change in the architecture

Four decisions, all made early, all cheaper than the cycle they avoid.

Narrow what the model is responsible for

Every piece of behaviour you place inside the model is behaviour that changes when you retrain. Every piece you place in deterministic code is behaviour that does not.

So the boundary between model and code is not only an engineering choice, it is a regulatory surface. Thresholds, unit conversions, range checks, disqualification rules for unusable signal, and safety interlocks all belong in code where they can be reasoned about, tested exhaustively, and changed under a much lighter argument.

The rule of thumb: if you can write it down as a rule, write it down as a rule. A model that has absorbed a rule it did not need to learn has taken a maintainable thing and made it a retraining dependency.

Reduce the causes of drift rather than the response to it

The usual drift response is monitor and retrain. In a device the better investment is upstream, in the things that make the input distribution move in the first place.

Device-to-device variance. Portable and point-of-care units are not identical, and differences between units, sensors, and acquisition conditions change the input in ways the system was never shown. Calibration and normalisation that remove unit-to-unit variation before the model sees the signal convert a retraining problem into a signal-chain problem, and the signal chain is easier to change.

Acquisition conditions. Placement, contact quality, ambient conditions. A front end that detects and rejects unusable acquisition removes an entire class of input the model would otherwise have to handle, and a refusal is a much cheaper failure than a wrong answer.

Population coverage at the start. A model trained on a narrow population will need retraining when it meets a wider one. Widening the population early is expensive; widening it after clearance is expensive and slow.

Make the evidence reusable

Much of the cost of a submission is assembling evidence. Much of that assembly is repeated work that was not designed to be repeatable.

Build the evaluation as a pipeline rather than as a study: a maintained dataset with a documented group split, a script that produces the full slice table, and versioned outputs tied to an artifact hash. Then a retraining event produces the same evidence in the same shape, and the comparison against the cleared version is mechanical.

The practice that matters most here is the split. A subject-disjoint split, holding out whole subjects rather than shuffling their samples, reports honestly. A random split over samples lets the same subject appear on both sides, which inflates the number and drives a performance claim that does not hold in the field. A claim built on an inflated number is the most expensive artifact in this entire process, because everything downstream is anchored to it.

Version the whole chain, not the weights

A device's behaviour is the model plus the preprocessing plus the constants plus the configuration. If you can change a normalisation constant without that showing up as a version change, then you have a path by which behaviour changes without the change being visible, which is precisely the thing the regulatory frame exists to prevent.

Hash the whole chain and treat any difference in the hash as a change requiring the same scrutiny, regardless of which component moved.

The question to ask before the first submission

"Which of the things most likely to change in the next three years are inside the model, and which are in code?"

Write the list. Input range changes as new sensor revisions ship. A new patient population. A new site with different acquisition habits. A threshold adjustment after field experience. A new device variant.

For each, ask what the response looks like. If the answer is retrain for most of them, the architecture has concentrated your future flexibility in the one component that is most expensive to change.

Moving even two of those out of the model and into deterministic code, or upstream into the signal chain, is often the highest-return architectural decision in the whole program, and it costs nothing at the point where it is cheap to make.

The failure this avoids

The characteristic bad outcome in medical device programs is not a rejected submission. It is a cleared device that cannot be maintained.

Performance drifts, the fix is a retrain, the retrain is a regulatory event nobody budgeted for, and the practical response is to leave the device as it is and manage the degradation. That is a worse clinical outcome than the fix, arrived at through a series of reasonable decisions, and it traces back to an architecture that put changeable behaviour in the least changeable component.

FAQ

Does retraining a model require a new FDA submission? It can. A modification to a cleared device that could significantly affect safety or effectiveness may require review before shipping, and retraining changes model behaviour including on inputs nobody tested. A Predetermined Change Control Plan is the pathway designed to allow specified changes without a new submission each time.

What is a Predetermined Change Control Plan? A plan submitted in advance describing what modifications will be made, how they will be validated, and within what bounds, so changes inside that envelope can be made without a new submission. It only covers what you described, so the flexibility you have later is set by what you wrote down early.

How should a regulated device architecture differ because of this? Put every behaviour you can express as a rule into deterministic code rather than into the model, reduce the upstream causes of drift through calibration and acquisition checks, build evaluation as a repeatable pipeline rather than a one-off study, and version the whole chain rather than only the weights.

Why does the train and test split matter so much in a device? Because a random split over samples lets the same subject appear on both sides, inflating the number and producing a performance claim that does not hold in the field. A subject-disjoint split reports honestly, and every downstream decision is anchored to that first number.

What is the worst outcome of getting this wrong? Not a rejected submission but a cleared device that cannot be maintained: performance drifts, the fix is a retrain, the retrain is an unbudgeted regulatory event, and the practical response becomes managing the degradation instead of fixing it.