Two percentiles are enough to fit the distribution, and every dependency dashboard has them. If you only have a mean, this is the wrong tool: the mean tells you almost nothing about a tail, and the tail is the entire question here.
| Rate | Per day | What it means | |
|---|---|---|---|
| One attempt exceeds the timeout | 0.316% | 1,263 | each one is a retry or a default |
| All attempts time out | 0.1579% | 632 | correlated and independent combined |
| All attempts error | 0.2000% | 800 | same path, different cause |
| If attempts were independent | 0.00001% | 0 | the number retries appear to buy |
| Fallback taken | 0.3579% | 1,432 | a default returned, silently |
How often one attempt exceeds the timeout
By where the timeout sits relative to the p99, across distributions of different tail weight. The lesson is in the columns rather than the rows: a timeout set at the p99 leaves roughly one percent regardless, but at half the p99 the same setting means two percent on a tight distribution and nearly nine on a heavy one.
| Timeout, as a multiple of p99 | Tight (p99 = 3x p50) | Typical (p99 = 8x p50) | Heavy (p99 = 20x p50) |
|---|---|---|---|
| 0.50x | 19.53% | 6.05% | 3.69% |
| 0.75x | 4.30% | 2.25% | 1.77% |
| 1.0x (at the p99) | 1.000% | 1.000% | 1.000% |
| 1.50x | 0.072% | 0.272% | 0.413% |
| 2.00x | 0.007% | 0.096% | 0.209% |
| 3.00x | 0.000% | 0.019% | 0.074% |
Amber is over half a percent of attempts, red over two percent. Every one of those attempts is a retry that loads the dependency further, or a default returned to a caller that cannot tell.
A worked example
A retrieval call behind a two second timeout, on a dependency with a 180 millisecond median and a 1.4 second p99, retried twice, at 400,000 requests a day. The team's reasoning is that two retries make a fallback essentially impossible, and on the standard arithmetic they are right: a 0.3 percent per-attempt failure rate cubed is about one in thirty million, which rounds to zero per day.
That arithmetic assumes the three attempts are independent draws. They are not. A dependency that is slow at 14:03 is slow for all three attempts, which arrive within a few seconds of each other. At fifty percent correlation the fallback rate is 0.36 percent, or about 1,432 requests a day, and those are orders of magnitude apart rather than a rounding difference.
Fourteen hundred a day matters for a specific reason: it is large enough to appear in an evaluation sample. A fifty-question evaluation drawn from production traffic will contain some of them, and each one looks like the model producing a bad answer from good context when in fact it received a default. That is a wrong answer nobody can attribute, and it quietly caps the accuracy anybody can measure.
The timeout also sits below the implied p99.9 of 2.75 seconds, so the slowest one in a thousand calls is cut off before it can answer. Raising it trades fallbacks for holding a connection longer while the dependency is stuck, which is a real trade rather than a free improvement. The change that is free is the counter.
The arithmetic, so you can check it
A lognormal is fitted through the two percentiles you supplied: sigma = ln(p99 / p50) / 2.326, with the median as the scale. The chance one attempt exceeds the timeout is then 1 - Phi(ln(timeout / p50) / sigma).
Attempts are combined as a mixture rather than as independent draws: rho x p + (1 - rho) x p^(attempts). At rho zero you get the textbook independence result, which is what most teams have in mind. At rho one every attempt shares one fate, which is what a degraded dependency actually looks like. Anywhere in between is a judgement, and the honest thing is that it is closer to one than to zero during exactly the incidents you care about.
Two limits worth stating. The lognormal is a reasonable default and real latency is often multimodal, with separate humps for a fast path, a cache miss and a retry, so a curve fitted through two percentiles will understate a bimodal tail. And the correlation figure is an input rather than a measurement: nothing here derives it from your data. Use the output to decide whether the counter is worth an hour, not as a substitute for the counter.
Why this failure survives is in your green dashboard is not evidence and before you blame the model. For whether those retries can even complete, use the timeout cascade check.
Questions
More often than the independence arithmetic suggests. Estimate it from your timeout against the dependency's latency distribution: fit a lognormal from the p50 and p99, compute the chance one attempt exceeds the timeout, then account for the fact that a slow period affects every attempt rather than one. The result is a lower bound, since errors and circuit-breaker openings take the same path.
Because attempts are not independent. The standard calculation treats a retry as a fresh draw, so three attempts at a one percent failure rate looks like one in a million. In reality a dependency that is slow at 14:03 is slow for all three attempts, so the effective rate is closer to the single-attempt rate than to its cube. Retries protect against transient blips and barely at all against a degraded period.
Because an error is loud and gets fixed. A fallback returns a value and a success status, so latency looks fine, the error rate looks fine, and the model appears to have produced a wrong answer when it was handed a default or never ran at all. Nothing downstream can distinguish the two, which is why the failure survives in production for months.
Count the real thing. The estimate exists to tell you whether the counter is worth an hour, and the answer is nearly always yes: a system where a third of a percent of requests return defaults is a very different object from one where none do, and most teams cannot say which they have. Once counted, the rate also becomes a leading indicator of dependency health.
It is a reasonable default and it is not the truth. Real latency distributions are frequently multimodal, with a fast path, a cache-miss path and a retry path producing separate humps, and a lognormal fitted through two percentiles will understate a bimodal tail. Use it to decide whether to measure, not as a substitute for measuring.