Private similarity search for each customer, served over an API. Every tenant has private data and defines its own criteria, and the system has to stay fast and cheap over ordinary HTTP and JSON. A cross-tenant leak here is a security failure: one customer seeing another's catalogue, a breach notice, and a contract that does not get renewed. Isolation done naively has its own cost. It multiplies spend per tenant until the business hits a hard ceiling while it is still small.
Similarity search turns items into points on a map, and a tenant boundary has to be drawn on that map too.
Similarity search starts with embeddings. A model reads each item in a catalogue and returns a long list of numbers that acts as its position on a very high-dimensional map. Items with similar meaning land near each other. To search, the query is placed on the same map and the nearest items are returned. Embedding the catalogue is the expensive step, and it only has to happen once per item. After that, answering a query is mostly a matter of looking up neighbours, which is fast and cheap.
Nearness on the map captures general similarity. It knows nothing about what a particular customer cares about this week, such as which attributes matter more, what to exclude, or how to break ties. So most production systems work in two stages. Retrieval pulls a shortlist of near neighbours quickly. Reranking then reorders that shortlist using the customer's own criteria. Because the criteria apply only at rerank, a customer can change them as often as they like, and the stored embeddings never need to be recomputed.
Serving many customers from one system raises the question of walls. There are broadly three shapes. Each tenant can get its own database, which gives clear separation and a cost that multiplies with every tenant. Tenants can share a store with each record tagged by owner, which is far cheaper and makes the tag filter the security boundary. Or something in between. In the shared shape, the decisive question is where the filter is enforced. If application code adds it to each query, one code path that forgets it is a leak. If it is enforced beneath the application, on every read, an application bug cannot remove it.
The visible feature is a comparison engine. Underneath it sit three harder requirements: keep each customer's data walled off, keep the cost of running it below what the feature earns, and survive the limits of the front end it plugs into. One of those limits was concrete and fixed. The front end automatically retries any API call that runs past 150 seconds, which makes idempotency mandatory on every endpoint whether anyone planned for it or not.
Multi-tenant features usually fail on cost or on walls, and the fix for one often causes the other.
The first instinct for isolation is physical separation: a database per tenant. It passes a security review easily and it works at small scale. Then each new customer adds a fixed slice of infrastructure cost and operational overhead, and the per-tenant spend climbs past what the feature earns. The design that felt safest becomes the ceiling on growth.
The cheaper instinct, a shared store with a tenant tag, fails differently. If the tag filter lives in application code, every query has to remember to include it. Most will. One code path, one hurried change, or one new endpoint will not, and that single omission is a cross-tenant leak. The cost problem gets solved by moving the security boundary into the layer most likely to have a bug.
The reflex, and the fix.
A separate database per tenant
It is the intuitive way to guarantee isolation, and it is easy to explain to a security reviewer.
It multiplies cost with every tenant and hits a ceiling early, so the architecture that felt safest becomes the reason the product cannot grow.
One shared store, filtered below the application
Enforcing the tenant filter on every read beneath the application layer, plus a rerank stage so user-defined criteria never force a re-embed.
Hard isolation, run cost held to a few dollars per tenant per month, and criteria that can change without recomputing anything.
Isolate below the application layer and put the variable criteria in rerank, because both costs grow with every tenant.
Embed the catalogue once
Embeddings are precomputed and stored, never generated per query. That costs cents to dollars, well below what most people picture when they imagine the bill for a similarity feature.
Retrieve, then rerank
User-defined criteria live in the rerank stage. A customer can change what they care about without forcing the catalogue to be embedded again.
Isolation enforced below the application
One shared store, with a tenant filter applied on every read beneath the application layer. A bug in application code cannot produce a cross-tenant read, because the application is not where the filter lives.
Feedback without a model per tenant
A global base ranker plus per-tenant weighting of features, falling back gracefully to the global ranker until a tenant has built up enough signal to be worth weighting.
Idempotent asynchronous endpoints
Because the front end retries anything past 150 seconds, a non-idempotent endpoint would fire twice under ordinary load. Here idempotency is a hard requirement.
What each choice cost, and what it bought.
AOne shared store, with the filter below the application
A database per tenant would have been the easier conversation with a security reviewer. It would also have put a per-customer cost on every new account and a ceiling on growth. Choosing a shared store meant the tenant filter became the security boundary, and so it was enforced beneath the application on every read, where no application bug can remove it. The cost is that the read path now has no acceptable failure rate and has to be tested as hard as anything in the system. The benefit is hard isolation at a run cost of a few dollars per tenant per month.
BPutting user-defined criteria in rerank
Folding each customer's criteria into the embeddings would have made retrieval alone return tailored results. It would also have meant re-embedding the catalogue every time a customer changed their mind. Keeping embeddings general and applying criteria to the shortlist adds a stage to every query. It means criteria can change freely at no compute cost, and the catalogue is embedded once.
CA global ranker with per-tenant weighting
Training a model per tenant would personalise strongly and multiply cost and maintenance with every customer. Instead, one global base ranker serves everyone, and each tenant's feedback adjusts the weighting of its features once there is enough signal to trust. New tenants start on the global ranker. They get sensible results from the start, and personalisation arrives when it has been earned.
DIdempotency as a requirement, set by the front end
The front end retries any call past 150 seconds, so every endpoint had to treat a repeated request as the same request. Making long work asynchronous and idempotent took design effort on each endpoint. Without it, ordinary load would have produced duplicate work and inconsistent results, because some calls will always run long.
A single headline number hides where a system fails. This work was scored on the dimensions that actually decide whether it holds in production, measured on real, held-out cases rather than the demo path.
A database per tenant that satisfies the security conversation, costs a multiple of what the feature earns, and hits its ceiling while the customer count is still small.
Private similarity search per customer over an API, with precomputed catalogue embeddings, a configurable rerank stage for user-defined criteria, hard tenant isolation enforced below the application layer, and idempotent endpoints. Shipped with zero cross-tenant leaks under adversarial testing and run cost held to a few dollars per tenant per month.
What it owns, and what it hands to a person.
Isolation is enforced on the read path, which is the right place for it, and that makes the read path a component with no acceptable failure rate. The per-tenant weighting needs accumulated signal, so new tenants run on the global ranker until they have earned weights of their own.
When a feature is priced per customer, the architecture is a cost decision before it is a capability decision. Put the expensive work where it happens once, put the variable work where it can change for free, and enforce the security boundary below the layer most likely to contain a bug.
In a shared multi-tenant store the tenant filter is the security boundary, so enforce it beneath the application where an ordinary bug cannot remove it.
When a feature is priced per customer, isolation by separate infrastructure can make cost grow faster than revenue.
Embed a catalogue once and apply each customer's criteria at a rerank stage, so preferences can change without recomputing anything.
Personalise with a shared base model and per-customer weighting, and fall back to the shared model until a customer has enough signal.
If a client retries slow calls automatically, every endpoint it calls has to be idempotent, whether or not anyone planned for it.