Your Camera Is Lying About Brightness

Mostafa Dhouib·10 min read

Your Camera Is Lying About Brightness

The short answer. Consumer cameras do not record light linearly. They apply a transfer function, a tone curve and other processing on the way to an output image, by design, so that the picture looks right on a display and uses the available bits where human vision is sensitive. The consequence for measurement is blunt: doubling the pixel value does not mean doubling the light that produced it. If your assay computes an intensity, a ratio, or a difference from encoded pixel values, part of what you are measuring is the camera's encoding. The fix is to linearise first, normalise against a reference in the same frame, and then choose a background subtraction method deliberately, because two standard methods disagree on the same image.

This one is invisible until somebody checks, which is why it survives into shipped products.

The symptom is not an obviously wrong number. It is a number that behaves oddly. The relationship between concentration and measured intensity is not quite linear in the way the chemistry says it should be, values drift between devices, and a standard curve fitted on one device does not transfer to another. Each of these gets patched locally and the underlying cause goes unexamined.

Why the nonlinearity is there on purpose

This is not a camera defect, and the framing matters because it changes the response.

Human brightness perception is roughly logarithmic. We discriminate fine differences in dark tones and coarse differences in bright ones. An image format with a limited bit depth that stored light linearly would waste most of its codes on bright values nobody can tell apart, and give too few codes to the dark values where perception is acute. So standard image encodings apply a nonlinear transfer function, allocating code values perceptually rather than physically.

On top of that, camera pipelines apply processing aimed at producing a pleasing picture. A tone curve that lifts shadows and rolls off highlights. Noise reduction that smooths exactly the low-level variation an assay may be trying to measure. Sharpening that creates overshoot at edges. Automatic white balance, which applies per-channel gains that change between frames. Automatic exposure, which changes the mapping from light to code value shot to shot.

Every one of those is correct for photography and hostile to measurement. The camera is not lying in the sense of malfunctioning. It is doing its job, and its job is not metrology.

The sensor underneath is approximately linear in photon count over most of its range. That linear quantity is what you want, and it is what the processing pipeline removes.

Linearisation

Undoing the encoding is the first thing to do and it has two routes.

Use raw capture where it is available. Raw data is close to the sensor's linear response before the transfer function and tone curve are applied. On platforms where raw is accessible, this removes most of the problem at the source and is worth considerable effort to arrange. It is not free. Raw files are large, demosaicing still has to happen somewhere, and black level and per-channel gain still need handling. But you are starting from a linear quantity.

Otherwise, measure the response and invert it. If you are stuck with encoded output, characterise the mapping from light to pixel value and apply its inverse before any arithmetic. The standard instrument for this is a calibrated step target with known reflectance or transmission steps. Photograph it under the capture conditions you actually use, plot pixel value against known relative intensity, and you have the response curve. Invert it with a lookup table.

A textbook inverse transfer function for a standard colour space is better than nothing and is not the same as measuring your own. It describes the colour space, not your device's pipeline. Measure the device you are shipping on.

Three details that cause real errors:

Subtract the black level before inverting. Sensors have an offset. Treating the offset as signal distorts everything, most severely at the dark end, which is frequently where the interesting measurements sit.

Lock everything you can lock. Exposure, ISO, white balance, focus, and any scene-adaptive processing the platform will let you disable. A response curve measured under automatic exposure characterises nothing, because the mapping changes per frame.

Linearise per channel. Channels do not share a response once white balance gains are applied. If your measurement uses a channel ratio, per-channel linearisation is the whole ballgame.

Normalise against something in the frame

Linearisation gives you a value proportional to the light that reached the sensor. It does not tell you how much light reached the sample, and that varies constantly. Ambient contribution, lamp ageing, device-to-sample distance, angle, shadowing from the operator's hand, colour temperature of the room.

The only robust answer is to put a reference in the same frame as the sample and measure relative to it.

A reference patch of known reflectance, positioned in the same plane as the sample so it receives comparable illumination. Measuring the sample as a ratio to the patch cancels a large share of illumination variation, because both are affected together.

More than one patch, spanning the range. A single white patch corrects a scale factor. Several patches at different known levels let you fit the response in situ, which catches a shifted tone curve as well as a lighting change. This is where the step target stops being a lab instrument and becomes part of the product.

A reference region on the consumable itself. If the sample sits in a cartridge or on a strip, printing reference patches onto it is usually the cheapest reliable option, and it gives you registration geometry as well.

Reject rather than correct when the reference is out of range. If the reference patches read outside their expected window, the capture conditions are outside what the correction was characterised for. The right response is to decline the measurement and ask for another capture. Extrapolating a correction beyond its characterised range is how a silently wrong result gets produced.

The general principle is that every absolute intensity measurement is a hostage to conditions you do not control, and every ratio against an in-frame reference is not. Design the measurement as a ratio wherever the chemistry allows it.

Why two background subtraction methods disagree

Having linearised and normalised, you still have to separate signal from background, and this is where a quiet source of disagreement lives.

Two methods are standard and both are defensible.

Local background estimation. Estimate the background from a neighbourhood around the signal region and subtract that. The rolling-ball style approach is the familiar version. The estimate follows slow spatial variation, so it handles uneven illumination and gradients across the frame.

Global or parametric background estimation. Fit a single surface, often a plane or a low-order polynomial, across the whole image, using regions known to contain no signal, and subtract the fitted surface. Or simply subtract a constant estimated from designated blank regions.

Run both on the same image and the corrected values will differ. The reason is not that one is implemented badly. It is that they make different assumptions about what background is.

Local estimation treats slow variation as background to be removed. If your signal region is large or has diffuse edges, part of the signal looks like slow variation at the scale the estimator is working at, and some of it is subtracted as background. The chosen neighbourhood size directly determines how much.

Global fitting treats only the fitted surface as background. It preserves diffuse signal that local estimation would eat, and it will leave behind any illumination structure the fitted model cannot represent. A plane fit cannot represent vignetting. A low-order polynomial cannot represent a shadow edge.

So the two methods partition the same image differently, and the gap between them is not an error. It is the ambiguity in the definition of background made visible.

What follows from that:

The method is part of the measurand. A reported intensity is only meaningful together with the background method and its parameters. Changing the neighbourhood size changes the reported value on identical images, which means it also silently invalidates the standard curve fitted under the old setting.

Pin it and version it. Method, parameters and the regions used, recorded with every result. A background method that changes with a library upgrade is a change to the assay.

Run both during development and look at the gap. If local and global estimation agree closely, the background is simple and the choice does not matter much. If they diverge substantially, your signal and your background are not well separated at the spatial scales involved, and that is a finding about the optics and the consumable layout rather than something to resolve by picking the method with nicer numbers.

The honest limit

Linearisation, in-frame normalisation and a pinned background method remove sources of error that are large and systematic. They do not make a phone camera into a spectrophotometer. Sensor noise, fixed pattern noise, spectral response that does not match your chemistry's absorption, limited dynamic range and the colour filter array's spatial sampling all remain, and some of those set hard limits on what the measurement can resolve.

This work also does nothing for precision. It removes bias and improves transferability between devices. The spread from repeated captures of an unchanged sample is a separate property, and a correctly linearised pipeline can still have a repeatability floor too wide for the decision you want. Characterise that separately before designing any threshold.

And platform behaviour is not stable. Camera pipelines change with operating system updates, parameters that could be locked on one release cannot always be locked on the next, and raw access varies by device. A response curve measured once is a snapshot that needs re-verification on the versions you ship against.

FAQ

Why isn't a pixel value proportional to brightness? Because standard image encodings apply a nonlinear transfer function on purpose, allocating the available code values according to human perception rather than to physical light, and camera pipelines add a tone curve and further processing on top. Doubling the pixel value therefore does not mean doubling the light. The sensor underneath is approximately linear, and the pipeline removes that property on the way to the output image.

How do I linearise a camera's response without raw capture? Photograph a calibrated step target with known relative reflectance or transmission under the exact capture conditions you ship with, plot pixel value against known intensity, and invert that curve with a lookup table applied before any arithmetic. Subtract the black level first, lock exposure and white balance, and do it per channel, since channel responses diverge once white balance gains are applied.

Is applying the standard inverse gamma curve enough? It is better than nothing and it is not the same thing. The nominal curve describes the colour space, not your device's pipeline, which adds a tone curve, local contrast handling, noise reduction and sharpening of its own. Measuring the response on the device and platform version you are shipping is what captures the actual mapping.

Why do I need a reference patch in the same frame? Because linearisation tells you how much light reached the sensor, not how much reached the sample, and that varies with ambient contribution, lamp ageing, distance, angle and shadowing. Measuring the sample as a ratio to a patch in the same plane cancels much of that, because both are affected together. Several patches across the range let you detect a shifted response as well as a lighting change.

Why do two background subtraction methods give different results on the same image? Because they define background differently. Local estimation treats slow spatial variation as background, so diffuse signal is partly subtracted and the neighbourhood size controls how much. Global or parametric fitting removes only the fitted surface, preserving diffuse signal but leaving structure the model cannot represent, such as vignetting under a plane fit. The gap between them is the ambiguity in the definition, not a bug.

Does getting the imaging chain right fix an unreliable result? Not on its own. Linearisation, in-frame normalisation and a pinned background method remove systematic bias and make values transfer between devices and conditions. They do not change the spread you get from repeated captures of an unchanged sample, which is a separate property that has to be measured before any decision threshold is designed.