Your Hybrid Search Is Silently Dense-Only, and Reciprocal Rank Fusion Will Not Tell You

Mostafa Dhouib·7 min read

Your Hybrid Search Is Silently Dense-Only, and Reciprocal Rank Fusion Will Not Tell You

The short answer. Hybrid search is supposed to be robust because the lexical side catches what the vector side misses. The standard recipe implements the lexical side with Postgres full-text search, and Postgres ships no text search configuration for Chinese, Japanese or Korean. So on a Chinese query the lexical branch returns zero rows. Reciprocal rank fusion handles that gracefully: it fuses one non-empty list with one empty list and produces the non-empty list's order. You get results, they are plausible, nothing errors, and you have been running dense-only retrieval while believing you are hybrid. I reproduced this on PostgreSQL 16.15 with pgvector 0.8.6.

Hybrid search has become close to a default, and for good reasons. Dense retrieval is strong on paraphrase and weak on exact tokens: part numbers, identifiers, names, rare terms. Lexical retrieval is the reverse. Running both and fusing the rankings covers more than either alone.

The recipe that implements this is copied widely enough to be effectively standard. A vector column with an index, a tsvector column with a GIN index, two queries, and reciprocal rank fusion to merge the rankings.

It has a failure mode that is specific, silent, and much more common than its visibility suggests.

Postgres has no Chinese text search configuration

Postgres full-text search works by parsing text into tokens and normalising them with a configuration that encodes the rules of a language. It ships with configurations for a list of mostly European languages.

Chinese is not on that list. Nor are Japanese or Korean. The reason is structural rather than an oversight: these languages do not delimit words with spaces, so tokenisation requires segmentation, which requires a dictionary and a model. That is a different class of component from a stemmer, and it lives in extensions such as zhparser or pg_jieba rather than in core.

If you call to_tsvector on Chinese text with a configuration that does not understand it, you do not get an error. You get a tsvector that is close to useless, and a query against it matches nothing.

Reciprocal rank fusion absorbs the failure

Here is where it goes from a gap to a silent one.

Reciprocal rank fusion scores each document by summing, over the result lists it appears in, one divided by a constant plus its rank. A document that appears in both lists scores higher than one appearing in either. That is the whole mechanism, and it is deliberately tolerant of lists of different lengths, including empty ones, because real retrieval systems return different numbers of results.

So when the lexical list is empty, fusion does not fail. It sums over one list. Every document's score is its dense contribution alone, the relative ordering is exactly the dense ordering, and the function returns normally.

The tolerance that makes fusion robust is the same property that makes this invisible. Nothing in the contract of the function is violated. There is no partial failure to detect, because from the function's point of view one branch returning nothing is an ordinary outcome.

Every test still passes

This is why it survives into production.

The tests people write for a search endpoint check that a query returns results, that the results are relevant, and that the latency is acceptable. All three pass. The dense branch is doing real work and returning real matches, so the output is plausible, often good.

What is missing is the specific class of thing the lexical branch exists to catch: exact-token matches that embeddings handle poorly. Part numbers. Invoice identifiers. Proper nouns the embedding model never saw. Those queries return something, because dense retrieval always returns something, and the something is semantically adjacent rather than correct.

So the symptom is not a failure. It is a quality ceiling that nobody can explain, on exactly the query class that hybrid search was adopted to fix.

How to check in one query

Do not evaluate the fused output. Evaluate each branch separately, and assert that the lexical branch returned a non-zero number of rows for a query it should match.

Concretely: take a document you know contains a distinctive exact token, query the lexical branch alone for that token, and count the rows. If it is zero, your lexical branch is not participating in any result that it is supposed to be improving, and you can stop debugging the fusion weights.

The general version of this, and the thing worth building into any multi-stage retrieval system, is per-stage instrumentation. Record how many candidates each branch contributed to each query. A branch contributing zero across a whole query class is then a visible number in a dashboard rather than a theory somebody has to think of.

If you need lexical search on Chinese text

There are three options. An extension that provides segmentation, such as zhparser or pg_jieba, which gives you a real text search configuration. An external search engine with proper CJK support. Or n-gram indexing, which is cruder and will work for exact substring matching without any linguistic knowledge.

Which one depends on what the lexical branch is for. If it exists to catch identifiers and part numbers, n-grams may be entirely sufficient and are far less operational work than an extension. If it exists to do actual language-aware search, you need the segmentation.

What is not an option is leaving it as it is and calling the system hybrid.

The honest limit

I reproduced this on PostgreSQL 16.15 with pgvector 0.8.6, against the standard recipe as it is commonly published. The behaviour follows from how text search configurations and fusion each work rather than from a bug in either, so I expect it to hold across versions, but I have measured it on that one.

The broader claim is narrower than it sounds. It is not that hybrid search does not work. It is that a hybrid system has two branches, either can silently contribute nothing, and the fusion step is specifically designed not to complain about that. Instrument the branches, not the output.

FAQ

Why does Postgres full-text search return nothing for Chinese queries? Because Postgres ships no text search configuration for Chinese, Japanese or Korean. Those languages do not separate words with spaces, so tokenising them needs segmentation with a dictionary rather than a stemmer, and that lives in extensions such as zhparser or pg_jieba. Calling to_tsvector with an unsuitable configuration does not error, it produces a tsvector that matches nothing.

Will reciprocal rank fusion warn me if one branch returns no results? No. Fusion sums one over a constant plus rank across the lists a document appears in, and it is designed to tolerate lists of different lengths including empty ones. With one empty list it returns the other list's ordering, normally and without error, because from the function's point of view an empty branch is an ordinary outcome.

How do I check whether my hybrid search is actually hybrid? Query each branch separately rather than evaluating the fused output. Take a document containing a distinctive exact token, query the lexical branch alone for that token, and count the rows. Zero means the lexical branch contributes to nothing. Better, instrument candidate counts per branch per query so a branch contributing nothing across a query class is visible as a number.

Why do my tests pass if the lexical branch is dead? Because they check that results come back, that they are relevant, and that latency is acceptable, and dense retrieval satisfies all three on its own. What disappears is the specific class the lexical branch exists for: exact tokens, identifiers, part numbers, rare proper nouns. Those queries still return something semantically adjacent, so the symptom is an unexplained quality ceiling rather than a failure.

What are the options for lexical search over Chinese text in Postgres? A segmentation extension such as zhparser or pg_jieba for a real language-aware configuration, an external search engine with proper CJK support, or n-gram indexing for exact substring matching with no linguistic knowledge. If the branch exists to catch identifiers rather than to do language-aware search, n-grams are often sufficient and much less operational work.

Does this only affect Chinese, Japanese and Korean? Those are where it is guaranteed, because no core configuration exists. The general pattern is wider: any branch of a multi-stage retrieval system can silently contribute zero candidates, and fusion will absorb it. A misconfigured language, an index that was never built, or a filter that excludes everything all produce the same silent degradation.