An extraction system was pulling structured data out of messy real-world documents at 44% accuracy. At that level a person re-checked nearly everything, so the automation saved almost no time. A wrong extraction is expensive. It lands in a record, gets copied into a report, and is found weeks later by someone who has to unpick everything built on it. A model that is confidently wrong on the hard cases is more dangerous than one that flags them.
A confidence score is only useful when it means what it says.
An extraction system reads a document, finds the fields it was asked for, such as a name, a date, an amount or a reference, and returns a value for each. Most models can also return a score alongside each value, a number meant to say how sure they are. Out of the box those scores are often poor guides. A model can report high confidence on outputs it gets wrong surprisingly often, and lower confidence on outputs that are fine. The score exists. It just does not mean anything a person can rely on.
Calibration is the property of a score meaning what it says. A calibrated system that reports high confidence is right that often, and one that reports low confidence is right correspondingly less. It is checked in a simple way: take a set of documents where the correct answers are known, group the outputs by the confidence the system gave them, and compare each group's stated confidence with how often it was actually right. Where the two disagree, the scores are adjusted until they line up. Then the check is repeated as new kinds of documents arrive, because calibration drifts.
Calibration is what turns accuracy into saved effort. Without a trustworthy score, errors are scattered invisibly among correct outputs, and the only safe review is a full one. With a calibrated score the outputs can be sorted. Confident extractions flow on, and doubtful ones are routed to a person. Where to draw that line is partly a business decision, trading review time against the cost of an error slipping through. Either way, the reviewer's work shrinks from everything to the doubtful tail.
The brief was to make the model more accurate. Accuracy alone was never going to rescue it. At any accuracy short of perfect, a reviewer still has to look at every output unless the system can say which ones to look at. What the automation lacked was self-knowledge: a way to tell the person which extractions were doubtful, so they could check those and leave the rest.
Extraction systems fail on the messy documents, and give no sign when they do.
Real-world documents are inconsistent. Layouts vary from sender to sender, scans are poor, fields move, labels change wording, and some documents simply omit what they should contain. An extraction system tuned on tidy examples does well on the tidy majority and badly on the rest. The trouble is that it does badly in the same confident voice it uses everywhere else.
That is why low accuracy and lack of trust compound each other. Once reviewers learn that some outputs are wrong and none are marked, they stop trusting any of them and check everything. At that point the automation is costing more than it saves, and improving the model a little does not change the reviewers' behaviour, because they still cannot see which outputs to skip.
The reflex, and the fix.
Push raw accuracy
It is what was asked for, it is measurable, and it feels like the obvious engineering response.
Even a large gain in accuracy leaves a reviewer checking everything, because without calibrated uncertainty there is no way to know which outputs to trust.
Make the system honest about its own doubt
Rebuilding the extraction path and adding calibrated uncertainty and a human review gate, beyond tuning for a higher number.
A 95% system where the 5% it is unsure about goes to a person, which is what makes the automation usable.
Fix the doubt as well as the accuracy, because the reviewer's time is what the automation was supposed to give back.
The extraction path, rebuilt
From 44% to 95% accuracy on messy real-world documents, in five days.
Output with calibrated uncertainty
Every extraction comes back with a confidence that means something, where before it came back as a bare answer the reader had to take on faith.
A human review gate on the doubtful tail
Low-confidence cases are caught before they flow downstream. The gate is what turns a model that is 95% accurate into a process where the 5% it is unsure about goes to a person.
What each choice cost, and what it bought.
ADiagnosing before rebuilding
A model-first reading of the problem would have scoped a long rebuild aimed at a higher number. Spending the first effort on understanding why reviewers checked everything showed that the missing piece was a doubt signal, with accuracy close behind. That diagnosis is what made a five-day fix possible. Its cost was resisting the urge to start retraining on day one. Its payoff was avoiding the multi-month rebuild a model-first reading would have scoped.
BRebuilding the extraction path on the real document mix
The extraction path was rebuilt and measured against the documents the system actually receives, messy ones included. Working on real documents is slower and less flattering than working on curated samples. It is the only version where the accuracy figure, 44 to 95% in five days, describes what the users of the system will actually see.
CMaking the confidence mean something
Adding calibrated uncertainty meant checking stated confidence against real outcomes and adjusting until the two agreed. That is extra work no user sees directly, and it has to be redone as documents change. It buys a score a reviewer can rely on, which is the only reason a reviewer would ever skip an output.
DPutting a human gate on the doubtful tail
Low-confidence extractions are stopped before they reach anything downstream and sent to a person. The system is therefore never fully unattended, and someone has to be available to clear the queue. In exchange the 5% of doubtful cases are caught where they are cheap to fix, and the reviewer's day changes from checking everything to checking the tail.
A single headline number hides where a system fails. This work was scored on the dimensions that actually decide whether it holds in production, measured on real, held-out cases rather than the demo path.
44 to 95% in five days is a measured, delivered result.
A more accurate model that a reviewer still checks in full, because nothing tells them which outputs to distrust.
44 to 95% extraction accuracy in five days, with calibrated uncertainty and a human review gate on the low-confidence tail, so review effort lands where the doubt is.
What it owns, and what it hands to a person.
Accuracy is measured on the mix of documents this system actually receives. Calibration has to be maintained as that mix shifts, and a confidence score that was honest last quarter can drift. The review gate also assumes there is a person available to take the flagged cases.
This is the second clean proof of a thesis that keeps holding: it is almost never the model. The brief named accuracy, the missing piece was self-knowledge, and the fix took five days against the multi-month rebuild a model-first reading would have scoped. The discipline carries anywhere a person reviews machine output. Make the system say what it does not know, and review effort collapses onto the part that needs it.
At any accuracy short of perfect, a reviewer has to check everything unless the system can say which outputs are doubtful.
A confidence score is only useful when it is calibrated, meaning outputs marked as sure are right about as often as the score claims.
Calibration drifts as the population of documents changes, so it has to be re-checked against known answers on a schedule.
Diagnosing why people distrust an automation can turn a multi-month model rebuild into a fix measured in days.
A human review gate on the doubtful tail is what turns an accurate model into an automation people are willing to rely on.