Multi-Tenant Retrieval: Leakage Is a Design Decision
Mostafa Dhouib··7 min readCross-tenant leakage in a retrieval system is not an accident that happens to careless teams. It is the default outcome of a design that treats tenancy as a filter applied at query time, because a filter is something that can be forgotten, bypassed, or applied one layer too late.
Multi-Tenant Retrieval: Leakage Is a Design Decision
The short answer. If tenant isolation is a filter passed alongside a query, then isolation holds only when every call site remembers to pass it correctly, forever, including the ones written under deadline by someone who joined last month. That is not isolation, it is a convention. Real isolation makes the wrong query impossible to express rather than merely incorrect, and there are three places to enforce it with very different guarantees.
The failure is well known and the framing usually is not. Teams treat cross-tenant leakage as a bug class to be careful about. It is better understood as the default behaviour of a particular design, which means the fix is architectural rather than diligence.
Why the filter approach fails
A filter is applied by the caller. Which means isolation depends on:
Every retrieval call site passing the tenant. Every new call site added in the next three years passing it. Every code path, including the batch job, the admin tool, the migration script, the debugging endpoint, and the analytics job. The filter being applied before the similarity search rather than after it. And the tenant identifier being derived from the authenticated session rather than from a request parameter the client controls.
Each of those is a place to be wrong, and being wrong is silent: a leaked passage looks exactly like a correct one to every part of the system, including the model that summarises it and the user who reads it.
Two of them deserve special attention because they fail even in well-run codebases.
Filtering after the search. Retrieve the top fifty by similarity, then drop anything belonging to another tenant. This looks correct and has two defects: the result quality degrades unpredictably, since a tenant whose documents are crowded out gets fewer results than requested, and the embedding of another tenant's content has already influenced the ranking. Worse, if the post-filter is ever omitted or reordered in a refactor, the failure is total and silent.
Tenant from a parameter. If the tenant id arrives in the request body rather than being derived from the authenticated session server-side, then isolation is enforced against a value the caller supplies, which is not enforcement.
Three places to enforce, with different guarantees
Physical separation: an index per tenant. The strongest. A query against tenant A's index cannot return tenant B's content because the content is not there. Leakage requires connecting to the wrong index, which is a coarse, loud, testable failure rather than a subtle one.
The costs are real: index proliferation, higher fixed overhead per tenant, slower onboarding, and harder cross-tenant operations such as global analytics. For a few hundred substantial tenants this is usually the right answer and teams talk themselves out of it on cost grounds that do not survive a serious look at the alternative's failure mode.
Enforced at the data layer. One index, with the tenant predicate applied by the storage layer itself rather than by the caller: row-level security, a scoped connection, or a query interface that structurally cannot express a cross-tenant query because the tenant is bound at connection time rather than passed at query time.
The property that matters: there is no way to write the wrong query. Not a rule saying you should not. No expressible form of it. That is the difference between a guarantee and a convention, and it is the pragmatic middle option for most systems.
Applied by the application. The filter approach. Everything above applies. It should be treated as a performance optimisation layered on top of one of the other two, never as the isolation mechanism.
What leaks besides documents
The part that survives the obvious fix and is missed most often.
Embeddings and the index structure. Approximate nearest neighbour indexes build structure from all the vectors in them. Depending on the index, timing and result patterns can reveal information about content the querying tenant cannot read. This is subtle, usually low severity, and worth knowing about before someone asks.
Caches keyed without tenancy. A query cache keyed on the query text alone returns tenant A's results to tenant B for an identical question. Identical questions across tenants are common, because people ask about the same policies and products. This is a total leak through a component nobody thinks of as part of the retrieval path.
Logs and traces. Retrieved passages logged for debugging, then exposed in a shared observability tool, in support tooling, or in an error report sent to a third party. The isolation held in the query path and failed in the telemetry path.
Evaluation and fine-tuning sets. Assembled from production traffic across tenants, then used to improve a shared model. That is a slow leak with the worst possible discovery story, because it is discovered at contract review.
Error messages and result counts. "No results found" versus "you do not have access" distinguishes non-existence from existence. Aggregate counts leak the same way.
Deciding, and proving it
Tenants who are competitors, or a regulated data class, or a contract with an isolation clause. Go physical. The cost conversation is easier than the alternative conversation.
Many small tenants, moderate sensitivity. Data-layer enforcement, with the tenant bound at connection time so a cross-tenant query has no expressible form.
Internal-only, single legal entity. Application-level filtering is defensible, with the caveat that internal-only has a way of becoming external later, at which point the design is load-bearing and expensive to change.
Then prove it, because a design that is correct on the whiteboard and unverified in production is a claim rather than a control:
A test that attempts a cross-tenant read on every retrieval path, running in CI, failing the build. Not a test that isolation works. A test that the violation is refused, on each path, by name.
Seeded canary documents in each tenant with distinctive content, and an automated check that they never surface for another tenant.
Cache keys asserted to include the tenant, tested rather than reviewed.
A log-scrubbing check that retrieved content does not reach shared observability.
The pattern is the same one that separates a real verifier from a named one: it is only a control if you can point at what it has rejected. For enterprise software sold to more than one customer, that evidence is also the thing a security questionnaire is actually asking for.
FAQ
How do you isolate tenants in a RAG system? Three options with different guarantees: a separate index per tenant, which makes leakage physically impossible; enforcement at the data layer with the tenant bound at connection time, so a cross-tenant query has no expressible form; or an application-level filter, which is a convention rather than a guarantee and should sit on top of one of the other two.
What is wrong with filtering results after the similarity search? Two things. Result quality degrades unpredictably because another tenant's documents crowd out the ones you wanted, and the ranking has already been influenced by content the caller cannot see. It is also totally and silently broken if the post-filter is ever omitted during a refactor.
What leaks besides documents? Query caches keyed without the tenant, which return one tenant's results to another for identical questions; logs and traces containing retrieved passages that reach shared observability or third parties; evaluation and fine-tuning sets assembled across tenants; and error messages or result counts that distinguish non-existence from lack of access.
When is a separate index per tenant worth the cost? When tenants are competitors, when a regulated data class is involved, or when a contract carries an isolation clause. Teams often talk themselves out of it on overhead grounds without seriously pricing the alternative's failure mode, which is silent and total.
How do you prove tenant isolation actually holds? With tests that attempt a cross-tenant read on every retrieval path and fail the build when it succeeds, seeded canary documents checked automatically against other tenants, asserted cache keys, and a check that retrieved content never reaches shared logging.
Split your failures into retrieved and not-retrieved in an afternoon, measure recall across k, score by slice, and check citation support mechanically. Ends in a seven-question hiring sheet with a scoring column.
Tell us the system, the stakes, and the date that matters. You get a straight technical reply from the person who would lead the work, within 24 hours.
Bring us the program