Industrial AutomationField engagement

The two ways to be wrong on a plant floor are nowhere near the same size, and a model tuned for accuracy is solving a problem the plant does not have.

Fractional CTO and lead engineer.
41%
fewer false alarms
87
unit fleet
cost-weighted
operating point
Missed failuredowntime, damage, safety
False alarma wasted inspection
The two ways to be wrong are not the same size, so the threshold is set by cost, not by accuracy.
What was at stake

On a plant floor there are two ways for a warning system to be wrong. A missed failure means unplanned downtime, secondary damage, a blown schedule and, at the limit, a safety incident. A false alarm means a wasted inspection. That is tolerable once. Repeated, it teaches the crew to ignore the system, and then a real warning arrives to people who have stopped listening. The second failure is the one that kills these programmes, because it is gradual and nobody records the day the system stopped being believed.

How this works

A machine usually announces a failure long before it fails, and predictive maintenance is the work of hearing it in time.

Most mechanical failures are not instantaneous. A bearing begins to wear, a shaft drifts out of alignment, a winding starts to run hot. Long before the machine stops, those changes show up as small shifts in vibration, motor current or temperature. Reliability engineers describe this with the P-F interval: the stretch of time between the point where a developing failure first becomes detectable and the point where the machine actually fails. Predictive maintenance lives inside that interval. A warning near its start leaves time to order parts and plan the work. A warning near its end is barely better than the failure itself.

Detecting the onset sounds like a classification problem, and that framing causes trouble. Failures are rare, so almost every reading is healthy, and a system that never raises a warning scores very well on accuracy while catching nothing. The useful way to think about it is as two error rates pulling against each other. Lower the alert threshold and you catch more real failures but raise more false alarms. Raise it and the false alarms fall but more failures slip through. Where to sit on that trade-off depends entirely on what each kind of error costs the plant, and those costs are almost never equal.

There is also the question of what normal means. A pump running hard under heavy load vibrates differently from the same healthy pump idling. If every reading is compared with a single average, each change of load or speed looks like an anomaly. Comparing a reading with what is normal for the current operating condition removes that whole class of false alarm, and it requires knowing how the equipment behaves as much as how the data looks.

  1. Machine running healthy
    readings sit within normal for this load and speed
  2. Failure begins to develop
    wear, misalignment or heat, not yet visible
  3. First detectable change
    the P point, where the window opens
  4. Condition indicator trends
    vibration or current moving steadily
  5. Warning with lead time
    early enough to plan the work
  6. Functional failure
    the F point, where the window closes
A sudden failure skips the middle of this sequence entirely, and nothing can warn on it.
FigureThe life of a developing failure, from the first detectable change to breakdown. The useful window for a warning lies between them, and a warning only has value if it arrives early enough in that window for the crew to act. Some failures have no such window at all.
The constraint

Accuracy is the wrong target by construction. Failures are rare, so a model that always reports healthy scores over ninety-nine percent and catches nothing. Two more problems sit underneath. The same healthy machine produces different vibration and current signatures at different loads and speeds, so a reading judged against one global baseline flags normal operation as anomalous every time the operating regime changes, and that is the dominant source of false alarms. And sensor data from a working plant is not clean. A dropout or a slow calibration drift can mimic a fault convincingly if nothing upstream conditions the signal.

alertgradualsuddenpredicted before it failstimecondition indicator
What degrades gradually can be predicted before the crossing. What fails suddenly leaves nothing to warn on.
Where it usually breaks

Predictive maintenance usually fails by being disbelieved, long before it fails by being wrong.

The common story is not a system that misses a dramatic failure. It is a system that raises too many alarms in its first weeks, each one checked and found to be nothing, until the crew stops checking. From then on the system is still running, and still correct some of the time, but nobody acts on it. When the real warning arrives, it is treated like all the others.

The causes of those false alarms usually sit upstream of the model. Readings judged against the wrong baseline, sensors that drop out or drift, and thresholds chosen to maximise a score all produce alerts that look like detections. A model can be correct about the data it was given and still be wrong about the machine, because the data was never conditioned or put into context.

01

The accuracy trap

Failures are rare, so a system that never alerts scores very highly. Optimising for accuracy rewards the behaviour that catches nothing.

02

The global baseline

Healthy readings at a different load or speed are compared with an overall average and flagged as anomalies. Every change of regime produces a false alarm.

03

The sensor that lies

A dropout or slow calibration drift looks like a developing fault. Without conditioning upstream, the system raises a confident alarm about a broken sensor.

04

The warning with no lead time

An alert that arrives just before failure is technically correct and practically useless, because there is no time left to plan the repair.

05

The slow loss of trust

Repeated false alarms teach the crew to ignore the system. Nobody records the day it stopped being believed, and the next real warning lands on deaf ears.

Judged against a global baseline
Machine moves to heavy load
vibration and current rise, as they should
Reading exceeds the overall average
the baseline knows nothing about load
False alarm raised
an inspection that finds nothing
Judged against its operating context
Machine moves to heavy load
vibration and current rise, as they should
Reading matches normal for this load
baseline for the current load and speed
No alarm, correctly
attention kept for real faults
Same machine, same reading
This decision depends on understanding the equipment, not only the dataset.
FigureThe single largest source of false alarms, and how it goes away. Judged against one global baseline, a healthy machine changing load looks anomalous. Judged against what is normal for the current load and speed, the same reading is unremarkable and the alarm never fires.
The fork

The reflex, and the fix.

Road not taken

Tune for accuracy, alert on anomalies

Pull

Standard, quick, and it produces a high score on historical data.

Why not

It either misses the rare failures or floods the crew with false alarms, and either way the system is switched off within a month.

Road taken

Cost-weighted, with actionable lead time

Accepted

Normalisation for operating context, a threshold tuned to cost, a lead-time gate, and calibration against real run-to-failure history. More to build and more to tune.

Bought

Warnings the crew believes, early enough to act on.

Decision

Optimise for the cost of the plant's errors and the timeliness of its warnings, not for a metric on a slide.

How it was built
01Conditioning
›
02Context normalize
›
03Health indicators
›
04Decision
›
05Evaluate
measured on real, held-out cases, then tuned
01

Condition the signal first

Denoise, fill and resample gaps, remove drift and align sensors to a clean baseline before anything else runs. Conditioning removes false signals that no downstream model can recover from. Half of what looks like a detection problem is a signal processing problem nobody addressed.

02

Normalise for operating context

Each reading is judged against the baseline for the current load and speed, not against one global baseline. This single decision removes the largest source of false alarms, and it depends on understanding the equipment as much as the dataset.

03

Predict only what leaves a trail

The system tracks condition indicators that trend as a machine degrades, and predicts the crossing before it happens. Part of its honesty is knowing which failure modes degrade gradually and can be seen coming, and which arrive suddenly and cannot.

04

A decision layer, not a score

An alert fires only when a cost-weighted threshold and a lead-time gate both agree. Both are calibrated on real run-to-failure history and held just short of the false-alarm rate that would cost the crew's trust. Everything else stays quiet, on purpose.

Inside the key decisions

What each choice cost, and what it bought.

ATuning to cost, not to accuracy

A threshold chosen to maximise accuracy is easy to justify on a slide. On a plant floor it is meaningless, because a missed failure and a false alarm cost wildly different amounts. Setting the operating point by the real cost of each error means the plant's economics have to be understood, and the threshold has to be revisited as they change. It buys an alerting system aimed at the plant's actual problem.

BConditioning and context before modelling

It is tempting to start with the model, because that is the visible and interesting part. Conditioning the signal and normalising for operating context first delays any model result. It also removes the largest sources of false alarms before a model ever sees them, which no amount of downstream modelling could have done.

CRequiring both a cost threshold and a lead-time gate

A single score crossing a line is the simplest possible alert. Requiring two conditions to agree, that the risk is worth acting on and that there is still time to act, means some warnings are held back. The ones that remain are warnings the crew can use, and every alert they act on successfully makes the next one easier to believe.

DBeing explicit about what cannot be predicted

Claiming coverage of every failure would have made the system sound more capable. Saying plainly that sudden and random failures leave no trail narrows the promise. It also means the crew knows which failures the system is watching for and which still need other safeguards, so nothing is assumed to be covered when it is not.

Condition the raw signal
denoise, fill gaps, remove drift
Normalise for operating context
the largest source of false alarms removed here
Track health indicators
only modes that leave a trail
Decide on cost and lead time
both must agree before an alert
Evaluate against real failures
calibrated on run-to-failure history
A model built on an unconditioned signal can be correct and still useless.
FigureWhere the decisions sit along the path from raw sensor to alert. The stages before any prediction is made are where most false alarms are removed. The decision stage fires only when cost and lead time agree.
How it was measured

A single headline number hides where a system fails. This work was scored on the dimensions that actually decide whether it holds in production, measured on real, held-out cases rather than the demo path.

Cost-weighted errorRecall on real failuresFalse-alarm rateLead-time distributionCalibration
too early
actionable window
too late
failure
vague, ignored
order the part, schedule the stoppage
no time to act
A correct warning at the wrong moment is useless. The target is the window where there is still time to act.
The gate
Do the cost-weighted threshold and the lead-time gate both agree?
Alert the crew
a warning worth acting on, with time to act
Stay quiet, deliberately
where false alarms fell 41%
Calibrated on real run-to-failure history, and maintained as the economics change.
FigureThe rule that set the operating point across the 87-unit fleet. An alert fires only when the risk justifies an inspection and there is still time to plan one. Everything else stays quiet, and that quiet is how false alarms fell 41% while the alerts that remained were believed.
figures

What it produces
Without this discipline

A high score on a slide and, on the plant floor, either silence on the failures that matter or a stream of false alarms that gets the system muted within a month.

This system

False alarms fell 41% across the 87-unit fleet, and the alerts that remained were believed and acted on, early enough to prevent the failures that mattered. The operating point was set by the real cost of each kind of error and not by a generic accuracy figure.

cost-weightedactionable lead timecontext-awarecalibrated for trust
The operating envelope

What it owns, and what it hands to a person.

Handled with confidence
Gradual degradation modes with a signal
Within trained operating conditions
With actionable lead time
Flagged for review
Low confidence
An operating regime not seen before
Out of scope by design
Sudden or random failures
Modes with no signal or no failure history
The honest limit

It predicts the degradation modes it has signal and history for. A truly sudden or random failure leaves nothing to warn on, and no system will change that. Conditions outside its training degrade it. The cost threshold has to be maintained as the economics and the equipment change. And it informs a maintenance decision. It does not make one.

What it generalizes to

Three of the five decisions that carried this work sit below the model: signal conditioning, operating-context normalisation and the choice of health indicators. A specialist hired only to build the model would have built a correct model on an uncorrected signal, judged against the wrong baseline, and would have been entirely within scope while the system failed. The discipline carries to any detection problem with rare, expensive events. Weight the errors by what they cost, normalise for conditions, demand actionable lead time, calibrate for trust, and be honest about what has no signal at all.

What to take from this

When failures are rare, accuracy is a misleading target. A system that never alerts can score very highly and catch nothing.

Set an alert threshold by what each kind of error costs, because a missed failure and a false alarm are almost never equally expensive.

Compare readings with what is normal for the current operating condition, or every change of load will look like a fault.

A warning is only useful if it arrives with enough lead time to act on. Measure lead time as carefully as detection.

The quickest way to kill a monitoring system is a stream of false alarms. Trust is lost gradually and is very hard to win back.

How we engage

You have a system like this one.
Tell us where it stands.

Whether it is failing, not yet built, or about to meet a scale it has never seen, we can tell you what we see.

Start a conversation
mostafa@opulion.dev · Response within 24 hours · By inquiry