What a Two-Week Diagnostic Should Deliver

Mostafa Dhouib·September 6, 2026·7 min read

What a Two-Week Diagnostic Should Deliver

The short answer. Five things: a named cause with the evidence that establishes it, the evidence in a form you can re-run yourself, a ranked list of everything else found, a fix scoped tightly enough to hand to a different vendor, and an explicit list of what was not examined. The last two are the ones that separate a diagnostic from a pitch, because a real finding survives being handed to someone else and a pitch does not.

"We will do a two-week discovery" is one of the least specified purchases in enterprise software. Two weeks later something arrives, it is usually a deck, and the deck usually recommends the engagement the vendor wanted to sell.

Here is the standard it should meet instead. This is written so you can put it in a statement of work before anyone starts.

The five deliverables

One: a named cause, with the evidence

Not "several factors are contributing." Not "the architecture would benefit from." One primary cause, named specifically enough that a fix follows from it.

What good looks like. "The training data is 78 percent from one collector. The test set was split at random, so that collector appears on both sides, and the model is scoring on data from the same room and equipment it was built around. The 95 percent measures internal consistency, not capability."

What that is not. "The model needs retraining with more diverse data." That is a recommendation dressed as a finding. It skips the part that would let you check it.

The test: could a competent engineer who has never seen your system read the finding and predict the symptom? If yes, it is a cause. If no, it is an observation.

Two: evidence you can re-run

The finding has to come with the artifacts that produced it, in a form your own team can execute after the engagement ends.

The query that sliced the training data by collector. The script that hashed the deployed artifact against the evaluated one. The diff of the preprocessed tensor between the two pipelines. The count of fallback-path invocations, and the code that produced the count.

This matters for a reason beyond verification. Almost every one of these artifacts is a permanent instrument. The fallback counter is not a one-time measurement, it is a metric you should have had for two years. A diagnostic that leaves those behind has raised your floor. One that leaves a PDF has not.

Three: a ranked list of everything else

A real investigation finds more than one thing. Some of it is unrelated to the presenting symptom and worth knowing anyway.

The list should be ranked by expected cost of leaving it alone, not by how interesting it is. Each entry needs three things: what it is, what it will cause if untouched, and roughly what it takes to fix.

The reason to demand ranking specifically is that it forces a judgement the vendor would otherwise avoid. An unranked list of twenty observations is a way of appearing thorough without committing to anything, and it transfers the prioritisation problem back to you, which is the part you were paying to have solved.

Four: a fix scoped tightly enough to hand to somebody else

This is the deliverable that separates a diagnostic from a sales document, and it is the one to write into the contract.

The recommended fix must be specified well enough that you could hand it to a different vendor, or your own team, and have them execute it without the diagnosing party involved.

That means: what changes, in which component, verified how, with what acceptance criterion, and what the cheapest experiment is that confirms the diagnosis before the full fix is built.

A vendor who does this is giving up their lock-in on purpose, and that is precisely the signal you want. A vendor whose fix cannot be handed on has produced a finding that only they can act on, which is a commercial position rather than a technical one.

Five: what was not examined

Every diagnostic has a boundary. Two weeks is not enough to examine everything, and pretending otherwise is the failure mode that lets a second cause survive undetected behind the first.

The deliverable should say plainly: these layers were examined, these were not, and here is why that was the right allocation. If the hardware was out of scope, say so. If only one of three data sources was sliced, say which and what it would take to do the others.

This is also how you avoid the worst outcome of a diagnostic, which is not a wrong finding but a right finding treated as the whole picture. A system with a data problem can also have a seam problem, and fixing the first without knowing about the second produces an improvement that stops short of the goal and nobody can explain why.

What should happen in the two weeks

Roughly, and in this order, because the order is most of the value: each phase is cheaper than the one after it and most investigations end before the last.

Note what is not in the table: building anything. A diagnostic that spends its second week starting the fix has quietly converted itself into the engagement it was supposed to inform.

Four questions to put in the statement of work

Will the deliverable name a single primary cause, with the evidence attached?

Will the evidence be re-runnable by our own team after you leave?

Will the recommended fix be scoped so that a different vendor could execute it?

Will the report state explicitly what was not examined?

All four are yes-or-no, all four are cheap to agree to if you intend to do them, and any hesitation on the third one tells you what the engagement actually is.

FAQ

What should an AI system audit deliver? Five things: one named primary cause with its evidence, the evidence in a form your team can re-run, a ranked list of everything else found, a fix scoped tightly enough to hand to a different vendor, and an explicit statement of what was not examined.

How long should a technical diagnostic take? Two weeks is a reasonable envelope for a production AI system: one to two days of plumbing checks, two to three on data distribution, one to two on evaluation integrity, two to three on seams, and two to write up and hand over. Anything longer is usually a build phase in disguise.

How do I tell a real diagnostic from a sales document? Ask whether the recommended fix could be executed by someone else. A real finding survives being handed to another vendor or to your own team. A finding that only the author can act on is a commercial position rather than a technical one.

Why does the report need to say what was not examined? Because the worst outcome of a diagnostic is not a wrong finding, it is a right finding treated as the whole picture. Systems commonly have a second cause behind the first, and fixing one without knowing about the other produces a partial improvement that nobody can explain.

Should the diagnostic team also do the fix? They can, but the scope should be written so they do not have to. Making the fix portable removes the incentive problem from the finding, and a vendor confident in their diagnosis loses nothing by it.