What is hybrid search and why is it better than pure vector search?

TL;DR

• Hybrid search combines lexical matching (like BM25) with vector similarity (embeddings) to get both exact term matches and semantic matches.
• It retrieves candidate sets using one or both signals, then merges or reranks them with a combined score $score = \lambda \cdot score_{lex} + (1-\lambda) \cdot score_{vec}$.
• Hybrid often improves recall for keyword-heavy queries and precision for semantic queries, while avoiding some vector-only failures.
• Key tradeoffs: higher indexing and query complexity, need for calibration of scores, and careful selection of candidate set sizes.

In this question, we will learn what hybrid search is, why practitioners prefer it over pure vector search in many production settings, and how to build a simple hybrid scoring pipeline you can explain in an interview.

We will cover the following:

• The intuition
• How it actually works
• Implementation patterns
• When to use each
• Tradeoffs and failure modes
• Questions the interviewer might ask
• What the interviewer is really testing

Direct answer: Hybrid search is a retrieval approach that combines lexical matching (for example BM25 or exact term matches) with vector similarity on embeddings, usually by retrieving candidates with one or both methods and then combining scores for a final ranking. It is better than pure vector search when you need both precise keyword handling and semantic understanding, because hybrid reduces missed exact matches and can improve precision for queries that mix keywords and intent.

The intuition (an analogy that makes it click)

Think of a library assistant who knows both the card catalog and the book summaries. The card catalog is fast at finding exact titles or authors (lexical). The summaries help find books on the same idea even if the words differ (semantic). Using both, the assistant first fetches likely books from the catalog and the summaries, then skims them together to pick the best ones. That combined approach is more reliable than relying only on summaries that might miss precise phrases or only on the catalog that misses synonyms.

How it actually works (the real mechanics, with a concrete worked example)

A common hybrid architecture has these steps:

1. Generate an embedding for the query, call it $q$.
2. Run a lexical search (for example BM25) to get top $N_{lex}$ candidates with scores $score_{lex}$.
3. Run a vector search to get top $N_{vec}$ candidates with vector similarities $score_{vec}$.
4. Union the candidate sets, compute a normalized hybrid score and rerank.

A typical hybrid scoring formula is:

$$score_{hybrid}(d) = \lambda \cdot score_{lex}(d) + (1-\lambda) \cdot score_{vec}(d)$$

where $0 \le \lambda \le 1$ calibrates the balance.

Worked toy example. Suppose we have three documents A, B, C and a query "install python package". We get these raw scores:

We normalize each column to $[0,1]$ before combining (min-max or z-score). After min-max normalization the numbers become:

Now set $\lambda = 0.6$ and compute hybrid scores:

$$score_{hybrid}(A) = 0.6\cdot 1.0 + 0.4\cdot 0.3125 = 0.725$$ $$score_{hybrid}(B) = 0.6\cdot 0.3636 + 0.4\cdot 1.0 \approx 0.618$$ $$score_{hybrid}(C) = 0.0$$

Final ranking is A, then B, then C. Document A had a strong exact keyword match that pure vector ranking would have missed as top result.

Implementation patterns

There are several practical patterns you can use, depending on latency and accuracy needs.

• Lexical-first: run BM25 for top $N$ candidates, then compute embeddings for those candidates and rerank by vector similarity (fast, good for keyword-heavy queries).
• Vector-first: run ANN search for top $N$ candidates, then optionally filter or rerank using lexical signals (useful when semantic recall is primary).
• Parallel retrieve and union: fetch top $N_{lex}$ and top $N_{vec}$ in parallel, union, and rerank (most robust, higher cost).
• Score fusion: compute normalized $score_{lex}$ and $score_{vec}$ and combine with a weight $\lambda$; optionally add supervised learning-to-rank on features.

Indexing considerations:

• Keep a lightweight inverted index for lexical matches.
• Keep a vector index (HNSW, IVF, FAISS) for embeddings of size $d$.
• Store precomputed metadata to speed up final ranking.

When to use each

• Use hybrid when queries mix keywords and intent, for domain-specific jargon, or when exact identifiers matter (product IDs, error codes).
• Use pure vector when queries are purely semantic, you cannot rely on stable tokenization, or you need language-agnostic matching and the dataset is clean of keyword traps.
• Use lexical-only if latency and memory are extremely constrained and semantics are secondary.

Tradeoffs and failure modes

Hybrid increases robustness but also complexity. You pay more in terms of storage (two indices), query latency (two searches in parallel or sequential), and calibration (score normalization and weight tuning). You must also handle contradictory signals where lexical and vector disagree strongly.

Questions the interviewer might ask:

Some follow-up questions you might get:

• How do you normalize scores from BM25 and cosine similarity?
  Use min-max, z-score, or rank-based normalization. Another option is to calibrate with a small labeled set and map raw scores to probabilities.
• How do you select $N_{lex}$ and $N_{vec}$?
  Start with modest values like $N_{lex}=100$ and $N_{vec}=100$, then measure recall and latency. Increase until marginal recall gain is small.
• How do you set $\lambda$ in the hybrid score?
  Tune on a validation set with relevance labels, optimize metrics such as NDCG or recall at $k$, or learn $\lambda$ in a simple logistic model.
• What vector index types work well in hybrid setups?
  HNSW and IVF+PQ are common for ANN search. Choose based on latency, memory, and accuracy tradeoffs, and ensure you can fetch vector distances for reranking.
• Can hybrid handle multi-lingual queries?
  Yes, if embeddings are multilingual. Lexical signals may need language-specific processing; hybrid can still help by combining both.
• How do you debug when hybrid returns poor results?
  Inspect the candidate lists from lexical and vector stages, check normalized scores, and examine examples where signals disagree.

Some things to note:

• Monitor per-query latency and candidate set sizes; large unions can blow up cost.
• Keep a labeled validation set covering keywords, paraphrases, and mixed queries for tuning.
• Consider supervised rerankers on top of hybrid features if you need extra precision.

What the interviewer is really testing

They want to see you understand both practical retrieval signals and tradeoffs: how lexical and dense representations complement each other, how to combine them sensibly, and the operational costs of doing so. They also want to know you can reason about calibration, candidate selection, and real-world failure cases.