An operator spent the working day editing images by eye: growing, removing or changing particular regions of pixels while leaving everything else exactly as it was. The work was repetitive and the judgement behind it was skilled, and it needed to become an automated system. The client ranked the failures plainly. A pixel changed that should have been left alone is worse than a hard image sent to a person. Reliability comes first, ahead of throughput and ahead of coverage.
Every automated image edit needs two masks, even when the tool only shows one.
Most image-editing automation is built around a mask: a map the same size as the image that marks which pixels an operation may touch. A detector produces the mask, the edit is applied inside it, and everything outside is left alone. That design carries one quiet assumption, which is that outside the mask and safe to leave are the same set of pixels.
In practice there are three sets. There are pixels the operation is meant to change, pixels it happens to leave alone, and pixels it must never touch whatever happens, because changing them is the error a customer notices. A single mask merges the last two. The common fix for a detector that undershoots makes this worse. Dilation grows the mask outward by a fixed amount so it covers the region fully, and it has no idea where the protected regions are. On any image where something protected sits close to the edit, dilation grows straight into it.
Splitting the job into a candidate mask and a protected mask makes the constraint explicit. The candidate says what might change. The protected mask says what must not. The final decision is derived from both, and the protected mask always wins. Because the protected region is now a real object, a test can check it directly: compare output to input, pixel by pixel, inside the protected region, and require the difference to be exactly nothing. That is a property you can prove for each image. A confidence score never gives you that.
The obvious build is a model that finds the region, with its mask grown by a fixed margin to be safe. That answers which pixels should change. The operator was answering a second question at the same time, without ever saying it aloud: which pixels must never change. That is a negative guarantee, and no amount of detector accuracy supplies it. A detector reports what it found and how confident it is. It makes no promise about what it left untouched.
Image automation goes wrong at the edges of regions, on the images nobody looked at.
Automated image processing is usually judged by looking at results. Someone scrolls through a sample, the edits look right, and the system is declared ready. The images that go wrong are disproportionately those where two regions meet, where the edit and something important are separated by a handful of pixels. They are a small share of any sample and a large share of the complaints.
The second pattern is that a model's error is reported as a rate, and a rate is the wrong unit for this kind of promise. A system that is right on nearly every image is still changing protected pixels on some of them, and the person it was meant to relieve now has to inspect every image to find out which. The automation has moved the work without removing it.
The reflex, and the fix.
Add a model and dilate the mask
Fast to demonstrate, and it looks correct on most images.
A fixed dilation grows straight across protected boundaries, and a model on its own can only report confidence. Neither can promise that a protected pixel came through unchanged.
Separate detection from modification, then prove the negative
Two masks in place of one, expansion that respects boundaries, deterministic rules applied over the model's output, and a pixel-level regression harness shipped as part of the product.
Zero unexpected changes on protected pixels, demonstrated on each image, with hard cases routed to a person.
Prove the negative, because that is the guarantee the operator was giving by eye every time they finished an image.
Unpacking what the operator actually did
Identify, modify, preserve looks like three steps. Written out properly it became eight separate judgements the operator was making at once, most of them never stated. Detection was then split from modification with a dual-mask design: a candidate mask of what might change, a protected mask of what must not, and a decision mask derived from both.
Expansion that stops at a boundary
Regions grow where growth is safe and are clipped cleanly against the explicit protected mask. The guarantee that unexpected changes number zero is checked and demonstrated on every image, never simply asserted for the system as a whole.
The model as a candidate detector
The model proposes. Deterministic rules enforce the exact pixel constraints. Validation decides whether the result is safe. The model keeps its place through measured reliability on the corpus, and a good demonstration earns it nothing.
Integration chosen around the engine
Four integration architectures were weighed: native, a plugin calling a separate engine, a hybrid, and headless. The deciding criteria were reliability and testability, with convenience for the first release a distant third.
A pixel-level regression harness
Every change passes an automated release gate: build, run the corpus, diff the output, check protected pixels, expected changes and performance, then pass or fail. Behind it sits a closed loop from production, where an edge case is traced to its root cause, added to the corpus, fixed and verified.
What each choice cost, and what it bought.
ATwo masks, with the protected one always winning
Defining protected regions explicitly is real work, and some are awkward to define. It would have been quicker to treat everything outside the detection as protected by default. That shortcut is exactly how dilation crosses boundaries unnoticed. Making the protected mask its own object is what allowed the guarantee to be tested at all, and a guarantee that cannot be tested is only a hope.
BDemoting the model to a candidate detector
The model's output is a proposal, and deterministic rules decide the exact pixels. That makes for a less impressive demonstration, because the model is visibly overruled at the boundaries. The benefit is that the product's error is no longer the model's error. A model on its own carries its error rate straight through to the customer. With rules enforcing the constraint, the model can be wrong about a candidate without any protected pixel paying for it.
CRouting hard cases to a person
Some images are sent for review and not processed automatically, so coverage is lower than a system that attempts everything. The client had already ranked the costs: a false positive on a protected pixel is worse than a review. Routing turns the expensive failure into the cheap one, and it is honest about where the system's competence ends.
DShipping the harness as product
Of the four integration architectures, the choice was made on whether the engine could be exercised automatically by the regression harness on every release. A harness that lives outside the product gets skipped under deadline pressure. One that sits in the release path cannot be skipped, and every edge case found in production becomes a permanent test.
A single headline number hides where a system fails. This work was scored on the dimensions that actually decide whether it holds in production, measured on real, held-out cases rather than the demo path.
A system that is right most of the time, cannot say which pixels it promises not to touch, and hands the job of catching its mistakes back to the person it was supposed to relieve.
A pipeline that guarantees zero unexpected changes on protected pixels and shows it on every image, sends hard cases to a person, and puts every change through a regression harness that ships as part of the product.
What it owns, and what it hands to a person.
The guarantee covers protected pixels. It says nothing about perfect detection and is not meant to. Hard images go to a person by design. The corpus also has to keep growing as new edge cases arrive, which is why the feedback loop is part of the architecture and not a process document somebody might forget to follow.
Where a wrong action costs more than asking a person, the deliverable is a negative guarantee. Separate detection from modification, enforce the constraint deterministically, and ship the regression harness as part of the product.
When a wrong change costs more than a manual review, the thing to build is a guarantee about what will not change, and that is a different problem from detecting what should.
A fixed dilation of a mask does not know where protected regions are, so it will eventually grow into one. Define protected regions explicitly and clip against them.
Treat a model as a source of candidates and let deterministic rules enforce hard constraints. The product's error then stops being the model's error.
Report reliability per image, not as a rate. A small rate of bad edits still forces someone to check every image.
Put the regression harness in the release path and feed every production edge case into its corpus, so each failure is fixed once and stays fixed.