What is a vector database, and how does it differ from a traditional database?

In this question, we will learn what a vector database is, how it differs from a traditional database, and when to pick one for an interview answer. We will keep the explanation practical and grounded with a concrete example.

We will cover the following:

• The intuition
• How it actually works
• Index types and performance
• When to use a vector database
• Tradeoffs and failure modes
• Questions the interviewer might ask
• What the interviewer is really testing

A vector database is a specialized store that saves high dimensional embeddings and provides nearest neighbor search for similarity queries, while a traditional database stores rows or documents optimized for exact lookups, joins, and transactional guarantees. Vector DBs focus on approximate similarity and retrieval speed for queries like "find items similar to this embedding," whereas traditional DBs focus on exact predicates and structured query semantics.

The intuition (an analogy that makes it click)

Imagine a library. A traditional database is like a catalog system where you search by exact fields: author name, ISBN, or subject code. You get exact matches and precise shelving rules.

A vector database is like a smell-based shelving system where each book has a scent profile that captures style or topic. When you bring a new book, you put it near books with similar scent. The shelving is fuzzy and based on similarity, so nearby books are not identical but related. That helps when you want "books like this one" rather than "books by this author."

How it actually works (the real mechanics)

Step 1: embed items. Each item is turned into a numeric vector using an embedding model. We store vectors of dimension $d$ for $n$ items.

Step 2: index vectors. Because scanning all $n$ vectors is expensive, we build an index that supports fast approximate nearest neighbor search.

Step 3: query by similarity. A query is embedded and the index returns the nearest vectors according to a similarity measure such as cosine or dot product.

A common similarity formula is cosine similarity:

Worked example: suppose $n = 10^6$ vectors and $d = 768$. A brute force search computes $n$ dot products costing $O(n d)$ operations. With $n=10^6$ and $d=768$ that is heavy for low-latency queries.

A vector index such as HNSW or IVF reduces the number of dot products dramatically at the cost of approximate results. The table below compares rough tradeoffs.

If we choose HNSW, typical query work is visiting a small fraction of nodes, not scanning all $n$ vectors. If we choose product quantization we can compress vectors and trade recall for memory.

Index types and performance

• HNSW (Hierarchical Navigable Small World) builds a graph where search hops follow edges toward neighbors. It often gives high recall and low latency, but memory overhead is larger because of auxiliary graph links.

• IVF (Inverted File) clusters vectors into $k$ cells and probes a few clusters at query time. It reduces work by narrowing to candidates, often combined with PQ (product quantization) for compression.

• Brute force scanning with optimized BLAS libraries or GPUs gives exact results but higher latency unless $n$ is small or hardware is powerful.

When listing complexity, use $n$ for items and $d$ for dimension. Brute force is $O(n d)$. Graph and cluster methods aim for sublinear behavior in $n$ at query time while adding index build cost and memory overhead.

When to use a vector database

Use a vector database when your queries depend on semantic similarity or perceptual closeness, for example:

• Semantic search over documents or passages.
• Image or audio similarity retrieval.
• Recommendation systems that match user embeddings to item embeddings.

Do not replace a transactional SQL database with a vector DB for tasks that need strong consistency, complex joins, or precise aggregations. In many systems you will use both: store canonical records in a relational store and embeddings plus index in a vector DB for retrieval.

Tradeoffs and failure modes

<Vector DBs are powerful but not magic. Design choices affect accuracy and cost.>

Common failure modes:

• Embedding drift: model updates change vector geometry and can invalidate indexes unless you reindex.
• Overcompression: aggressive quantization lowers memory but reduces recall.
• Poor metric choice: using Euclidean distance when cosine is the right measure can change results.

Questions the interviewer might ask

Some follow-up questions you might get:

How do you measure recall for an approximate index? Measure recall by comparing the index results against a ground truth brute force top-$k$ on a representative query set. Report recall at different $k$ and latency percentiles.

What similarity metrics are common and when to choose them? Cosine similarity and dot product are common for embeddings from neural models. Euclidean distance is used when magnitude matters. Choose the metric consistent with how embeddings were trained.

How do you handle updates and deletes? Small updates are possible but many indexes prefer batch rebuilds for high quality. Some implementations support dynamic inserts and lazy deletes with periodic compaction.

Can a vector DB replace a search engine like Elasticsearch? Not entirely. Vector DBs handle semantic similarity well. Search engines offer full text parsing, scoring pipelines, filters, and aggregations. Many systems combine both approaches.

How do you scale to billions of vectors? Use sharding, compressed indexes (PQ), and hybrid architectures. GPU acceleration and distributed HNSW or IVF clusters help handle massive $n$ at low latency.

Some things to note:

• Always validate with representative queries and measure recall/latency tradeoffs.
• Embedding maintenance and versioning are operationally important.
• Use hybrid filters (exact metadata filters plus vector search) to combine strengths.

What the interviewer is really testing

They want to know you understand the difference between exact, schema-driven data access and approximate, geometry-driven retrieval. They are testing familiarity with embedding concepts, nearest neighbor indexing, and practical tradeoffs such as recall, latency, memory, and operational concerns like reindexing and metric choice. Give an answer that shows when to use a vector database alongside traditional databases, not as a wholesale replacement.