Your Mel Spectrogram Is Not the Same as Theirs: Six Parameters That Have to Match

Mostafa Dhouib·7 min read

Your Mel Spectrogram Is Not the Same as Theirs: Six Parameters That Have to Match

The short answer. Two libraries asked for a mel spectrogram with the same sample rate, the same window size and the same number of bands will not necessarily give you the same array. At least six parameters differ in their defaults, and one of them is the definition of the mel scale itself, which has two established formulations in common use. A mismatch between the training feature path and the device feature path produces a systematic tilt across the spectrum rather than random noise, so the model degrades coherently and the symptom reads as a robustness problem. Pinning all six converts a bug that keeps coming back into a bug that is fixed once.

A team fixes a feature mismatch. The detector improves. Six months later the detector degrades again, somebody investigates, and it is a feature mismatch.

The second one is not a recurrence of the first. It is a different parameter in the same class, and the class has at least six members. Fixing them one at a time means fixing them forever.

The six

The mel scale formula. There are two established formulations in common use. One is a smooth logarithmic curve, the other is linear below a breakpoint around 1 kHz and logarithmic above it. They disagree about where every band boundary sits. Libraries differ in which they default to, and some expose a flag that is easy to miss. This is the one that surprises people, because the scale is usually assumed to be the one thing that is standard.

Normalisation of the filterbank. Filters can be normalised so each has unit area, or left unnormalised. This changes the relative weighting of bands systematically, with the effect strongest where filter widths vary most, which is across the whole upper range.

Whether the spectrogram is power or amplitude. Squaring the magnitude before applying the filterbank is a different operation from applying it to the magnitude directly, and the difference is not a constant factor once a logarithm follows.

Centring. Whether frames are centred on their timestamps, with the signal padded at both ends, or whether the first frame begins at sample zero. This shifts the entire time axis by half a window and changes the frame count.

Padding mode. When frames are centred, the signal is extended at the edges, and reflection, zero-filling and edge replication produce materially different content in the first and last frames.

Whether the filterbank is built in mel space or in frequency space. Constructing triangles that are symmetric in mel space and converting, versus converting band edges and building triangles symmetric in frequency, gives different filter shapes.

Every one has a defensible default. Every library chose independently.

Why the result is a tilt rather than noise

This is the part that makes the symptom misleading.

A random difference would show up as degraded performance everywhere, roughly uniformly, and would be recognised as a data problem quickly.

These differences are structured. A scale-formula mismatch moves band boundaries progressively more as frequency rises. A normalisation mismatch weights bands by their width. Both produce a smooth, monotonic difference across the spectrum: a tilt.

The model, trained on one tilt and run on another, sees inputs that are coherently wrong rather than noisy. It still produces confident outputs. Performance drops, often not catastrophically, and the drop is concentrated in whichever part of the spectrum carries the discriminative information.

So the symptom is a detector that works less well than it did in evaluation, with no errors and no obvious cause. That reads as a robustness problem, and the team starts improving the model, which is work aimed at a cause that is not there.

How to check

Do not compare model outputs. Compare the features directly, which turns a diffuse investigation into a definite answer in an afternoon.

Take one audio file. Compute the feature array with the training path. Compute it with the device path. Compare the arrays element by element.

If they match to numerical precision, the feature path is not your problem and you can stop looking at it. If they do not, the difference pattern identifies the parameter: a frequency-dependent ratio points at the scale formula or the normalisation, a constant offset in log space points at power versus amplitude, a shift along the time axis points at centring, and a difference confined to the first and last frames points at padding.

Then write that comparison as a test and run it in continuous integration. The cost is one audio fixture and one assertion, and it converts this entire class from recurring to closed.

Where the two paths come from

The mismatch is usually structural rather than careless.

Training happens in a research environment with a convenient feature library, chosen for expressiveness. Inference happens on a device, in a different language, with a different implementation chosen for size or speed. Nobody intends them to differ, and nothing connects them.

A single specification of the feature path, written down and referenced by both implementations, is the organisational fix. A test comparing the two is the engineering one. Both are cheap and neither tends to exist until after the first incident.

The wider lesson about attribution

A missed detection has several possible causes that look identical from outside. The signal was removed by a transform before it arrived. The threshold was calibrated on clean material and is wrong for the material actually seen. Or the feature path drifted.

Those have completely different fixes, and improving the model helps with none of them.

The harness that separates them applies one controlled condition at a time and measures the detection rate under each, so robustness becomes a curve against named conditions rather than a single number. Build that before improving anything, because an improvement aimed at an unestablished cause cannot be evaluated even in retrospect.

The honest limit

Pinning the feature path removes one cause. It will not make a detector robust to a transform that removes the signal, and some transforms do exactly that, in which case no feature path and no model recovers what is no longer present. Published work on audio watermarking has shown detection performance collapsing toward chance under certain neural codecs, which is a ceiling rather than a defect.

Knowing which of your misses are a ceiling and which are a bug is the actual deliverable. The feature comparison tells you cheaply whether one specific, common and entirely avoidable cause is in play.

FAQ

Why do two libraries give different mel spectrograms for the same audio? Because at least six parameters differ in their defaults, including the mel scale formula itself, which has two established formulations in common use. Filterbank normalisation, power versus amplitude, frame centring, padding mode and whether filters are built in mel or frequency space all vary as well.

What does a feature-path mismatch look like at inference time? A systematic tilt across the spectrum rather than random noise, because the differences are frequency-dependent and smooth. The model produces confident outputs on coherently wrong inputs, performance drops without errors, and the symptom reads as a robustness problem rather than a data one.

How do I check whether my training and device feature paths match? Compute the feature array for one audio file through both paths and compare element by element. Do not compare model outputs. A frequency-dependent ratio points at the scale or normalisation, a constant log offset at power versus amplitude, a time shift at centring, and a difference only in the first and last frames at padding.

Why does the same feature mismatch bug keep coming back? Because it is a class with at least six members, not a single bug. Fixing one parameter leaves the other five, and the next incident is a different member of the same class. Pinning all six and asserting the comparison in continuous integration closes the class.

Why do two feature paths exist in the first place? Training usually happens in a research environment with a feature library chosen for expressiveness, and inference happens on a device in a different language with an implementation chosen for size or speed. Nobody intends them to differ and nothing connects them, so the drift is structural rather than careless.

If the features match, why is my detector still missing? Two other causes produce identical symptoms: the signal was removed by a transform before arrival, or the threshold was calibrated on clean material and is wrong for the material actually seen. A harness applying one controlled condition at a time separates them, and some transforms destroy the signal entirely, which is a ceiling rather than a bug.