A neuro-wearable was days from a locked bill of materials. It captures sensitive analog EEG and runs a drainage protocol on which patient safety depends. The work was scoped as firmware. Three things raised the stakes. Patient safety makes correctness non-negotiable. Analog EEG is fragile enough that mistimed digital activity corrupts it. And a manufacturing clock means a catch is only worth something if it lands before the freeze.
Before the freeze a hardware fix is a drawing change. After it, the fix is a new board.
Hardware products pass through a point called the BOM freeze. The bill of materials, the full list of parts together with the board they sit on, is locked so that purchasing, tooling and manufacturing can begin. Before that point a design error is corrected by editing a schematic. After it, the same error is built into every unit, and correcting it means a board respin: a new layout, new prototypes, new validation, and a schedule that moves by however long all of that takes. The cost of a hardware fix does not rise gradually. It steps up at the freeze.
Many of the faults that cross that line sit in the seam between firmware and board. Firmware assumes a pin is wired a certain way, that a signal means what the documentation says it means, that a detection circuit reports what it was drawn to report. The schematic is drawn by someone else, on the assumption that the firmware will cope. Each side is correct on its own terms, and the fault exists only where they meet. A firmware engineer who reads the schematic as a reference will use it. One who reads it as part of the system will question it.
A device like this also mixes two kinds of electronics that dislike each other. Analog EEG is a tiny voltage that has to be amplified cleanly. Digital activity, a processor waking, a bus moving data, a regulator switching, draws current in sharp edges that couple into nearby analog circuitry through shared power, shared ground and plain proximity. If a conversion happens while a burst of digital traffic is underway, the noise lands in the measurement and looks like signal. Mixed-signal firmware therefore schedules time as carefully as it schedules logic, keeping the noisy work out of the windows in which the analog path is listening.
There is no model anywhere in this system. Everything that mattered was circuits, timing and safety logic, and none of it would have been improved by adding one.
- Schematic drawnby the board designers, to their own specification
- Firmware written against iton the assumption the documents are right
- Schematic audited across the seama fix here is a drawing change
- Bill of materials freezesthe cost of a hardware fix steps up here
- Boards fabricatedany remaining fault is now in every unit
- Fault found after the freezerespin, revalidate, reschedule
The contract said firmware. The risk lived one layer below, in the seam between the firmware and the board. An undocumented routing issue sat in the controller, and a charger-detect front end, if built as drawn, would become a defect fabricated into every unit the moment the bill of materials locked. An engineer working strictly inside firmware never opens those pages. The move that mattered was auditing schematics that were, strictly speaking, not the assignment.
Faults between hardware and firmware hide in what each side assumes about the other.
On a programme split between board designers and firmware engineers, each group verifies its own work against its own specification. The schematic passes review. The firmware passes its tests on an early board. What nobody verifies is the contract between them: that a pin really routes where the firmware expects, that a detection circuit produces the signal the code reads, that one side's assumption is actually the other side's design.
Safety-critical firmware fails in a related way. Code written for the expected sequence of events behaves well on the bench, where events arrive in the expected order. In use, a charger is connected partway through a cycle, a supply dips, an interrupt lands at an awkward moment, and a path nobody scripted runs. Interlocks exist to hold under exactly those conditions, and their guarantees decay unless something re-checks them every time the code changes.
The reflex, and the fix.
Stay inside firmware scope
It was the assignment and the contract, and nobody would have called it a failure.
The defect sat one layer below the scope line. Staying inside it would have delivered correct firmware onto a board with a permanent fault, found after the freeze, when the only fix is a respin.
Audit the schematic before the freeze
Working outside the contracted scope, on a manufacturing clock, with no guarantee that anything would turn up.
An open-ended risk of downstream delay and billing turned into a catch made before the freeze.
Open the layer below the scope line. The line marks the contract, and the risk does not respect it.
The audit that was not the assignment
A review of the schematic covering the controller routing and the charger-detect front end, done before the bill of materials locked and the trap became permanent.
Interlocks around the safety-critical path
The drainage protocol is wrapped in software interlocks built to hold under conditions nobody scripted. Patient safety leaves no room for a path that only behaves when events arrive in the expected order.
Verification on every change
Continuous-integration testing re-checks the safety guarantees each time the code changes, so a guarantee proven early cannot quietly lapse later.
Timing that protects the analog path
Digital activity is sequenced so it never lands on the analog EEG path at the wrong moment. That path is fragile enough that ordinary bus traffic at the wrong time degrades the very signal the device exists to capture.
Notes that keep the team moving
Findings were written up as formatted engineering notes, so the rest of the hardware team could act on them in parallel without waiting on the next finding.
What each choice cost, and what it bought.
AReading the schematic as part of the job
The contract covered firmware, and staying inside it would have been entirely defensible. Auditing the board design meant spending time outside scope, on a manufacturing clock, with no guarantee of finding anything. It found an undocumented routing issue and a charger-detect front end that needed correcting, both before the freeze. Had the audit not happened, the same findings would have arrived later as a respin, with delay and billing nobody could bound.
BInterlocks designed for the unscripted case
The drainage protocol could have been written to the expected sequence of events and tested against that sequence. It was wrapped in interlocks meant to hold whatever order events arrive in. That is more design work and more states to reason about. It bought a safety path whose behaviour is defined in the conditions nobody planned for, which is where patient-safety faults tend to live.
CVerifying the safety path continuously
Testing a safety guarantee once, at the end, proves it held on the day it was tested. Running the checks in continuous integration on every change costs build time and test upkeep. It means a later edit that quietly breaks an interlock fails the very build that introduced it, while the change is still small and fresh in someone's mind.
DTreating time as a design resource
Digital activity could have been left to run whenever the firmware found it convenient, with the analog path cleaned up by filtering afterwards. The firmware sequences digital work around the analog EEG path so the two do not overlap at the wrong moments. That constrains how the rest of the firmware is scheduled. It keeps noise out of the measurement at the source, which filtering cannot fully do once noise and signal share a sample.
A single headline number hides where a system fails. This work was scored on the dimensions that actually decide whether it holds in production, measured on real, held-out cases rather than the demo path.
Outcome is client-attested. The testimonial on this page is the only one on the site.
Correct firmware, delivered on schedule, onto a board with an undocumented routing fault and a charger-detect front end that becomes a fabricated-in defect at the freeze. It surfaces downstream as a respin and a bill with no ceiling.
The routing trap caught and the front end corrected before the bill of materials locked. Patient-safety firmware delivered with interlocks and continuous integration. Clean EEG timing under real digital load.
What it owns, and what it hands to a person.
Working with Mostafa has been an absolute masterclass in embedded engineering. He is not just writing firmware, he is actively protecting and elevating our entire hardware architecture. During the pre-manufacturing phase, Mostafa proactively audited the PCB schematics and caught a critical hardware-firmware integration trap, identifying an undocumented MCU routing issue and correcting the charger-detect front end, right before we locked the BOM. His foresight saved us from massive downstream delays and open-ended billing risks. His approach to patient-safety firmware is exceptionally rigorous. He tackled our complex drainage protocol by building bulletproof software interlocks, backing them up with continuous integration testing, and flawlessly choreographing the timing to protect the integrity of our sensitive analog EEG data. Mostafa communicates with extreme clarity, frequently providing perfectly formatted engineering notes to keep the rest of the hardware team completely unblocked. I have already issued him a bonus for his stellar performance, and his exactness has only increased as the scope has grown. If you need a top-tier embedded engineer who thinks three steps ahead and runs an incredibly tight ship, Mostafa is the gold standard.
Client review, verbatim. Public and attested.
This is an ongoing engagement, and the outcome that matters here is attested by the client and not self-reported. The specific controller part is deliberately not named, and the device is described only as far as the work requires.
The value came from refusing the scope line. Risk in a layered system concentrates in the seams, so an engineer who works only inside the assignment cannot own the risk, however well the assignment is done. The same holds for any programme that runs from hardware to software: find the layer below the one you were hired for, and check it while checking is still cheap.
The cost of a hardware fault steps up sharply when the bill of materials freezes. Review the board while a fix is still a drawing change.
Faults between firmware and hardware live in the assumptions each side makes about the other. Someone has to read both sides as one system.
A scope line marks a contract, not the location of the risk. The most valuable work on a programme is often one layer below the assignment.
Safety logic has to hold in the sequences nobody scripted, and its guarantees have to be re-checked on every change or they quietly lapse.
On mixed-signal devices, timing is part of signal integrity. Keep noisy digital work out of the moments when the analog path is measuring.