What are embeddings in the context of AI engineering?

In this question, we will learn what embeddings are, how they connect to vector-db systems, and why a software engineer building retrieval systems must be comfortable with the tradeoffs. We will use concrete examples and a formula or two so you can answer clearly in an interview.

We will cover the following:

• The short direct answer
• The intuition
• How it actually works with an example
• Indexing and search options compared
• Tradeoffs and failure modes
• Questions the interviewer might ask

Embeddings are dense numeric vectors produced by an encoder model that map inputs into a continuous space where semantic similarity corresponds to geometric proximity; vector databases store and index those vectors so you can perform efficient nearest neighbor search for retrieval, clustering, or downstream tasks. This lets you compare meaning rather than exact tokens, and you can tune the system by choosing encoder, vector dimensionality, and index structure.

The intuition (an analogy that makes it click)

Think of embeddings like coordinates on a map. Locations that are similar in meaning end up near each other. If you want restaurants that match "spicy vegan food," you look for points near the coordinates for that concept rather than matching exact words.

A vector database is the filing cabinet that holds the map and provides a fast way to find the nearest points. The encoder is the cartographer who chooses how to place things on the map.

How it actually works (the real mechanics, with one concrete worked example)

At a high level:

1. The encoder takes an input and produces a vector $v$ in $\mathbb{R}^d$.
2. The vector-db indexes those vectors and supports queries like "find the top k nearest vectors to $q$."
3. The retrieval is based on a similarity metric such as cosine similarity.

The cosine similarity between two vectors $u$ and $v$ is

Concrete example: suppose we embed three documents and a query using a small $d=4$ toy encoder. The vectors are:

Query: "how to bake apple pie" -> Query vector [0.88, 0.12, 0.0, 0.0]. Compute cosine similarity and you will retrieve Doc A then Doc B because they are nearest.

When we scale, $d$ may be 384, 1024, or larger. The vector-db will use indexing structures so that query time is sublinear in the number of vectors.

Indexing and search options (comparison table)

Different index types have tradeoffs in speed, memory, and recall. Here is a compact comparison:

Choice depends on dataset size $n$, required latency, and available memory.

When to re-embed and how to handle versioning

Embeddings change when you switch encoders or update training data. We recommend adding a version tag to each stored vector and supporting background re-embedding. For large corpora, do rolling re-embedding: process high-value documents first and use a hybrid search when both old and new embeddings exist.

You can also store the original text alongside vectors so you can re-embed offline and verify retrieval before swapping indexes.

Tradeoffs and failure modes

• Quality versus cost: larger, fine-tuned encoders produce better semantic placement but require more compute and storage.
• Index staleness: adding many upserts can fragment an index and harm recall until you rebuild or reindex.
• Metric mismatch: if embeddings are trained with a different objective than your retrieval use case, nearest vectors may not match user intent.

Questions the interviewer might ask

Some follow-up questions you might get:

How do you choose embedding dimensionality? Higher dimensionality can encode more nuance but increases storage and compute. Start with common sizes like $384$ or $768$ and test recall and latency.

When would you use cosine similarity versus dot product? Use cosine similarity when vector norms vary and you care about angle. Dot product can be fine when vectors are normalized or when models were trained with dot product objectives.

How do you handle multimodal embeddings? Use a shared embedding space or modality-specific encoders mapped into a common space. Evaluate cross-modal retrieval with held-out pairs.

What is approximate nearest neighbor search and why use it? Approximate methods trade small recall loss for large speed and memory gains. For large $n$ they are necessary to achieve low latency.

How do you test embedding quality? Use retrieval metrics like recall@k, MRR, and downstream task performance. Also inspect nearest neighbors qualitatively.

How would you handle privacy or PII in embeddings? Avoid embedding raw PII if possible. Consider techniques like redact-before-embed, differential privacy during model training, or encryption of source text.

Some things to note:

• Always version embeddings and index strategies.
• Combine embedding search with a reranker or filtering step when precision matters.

What the interviewer is really testing

They want to know you understand embeddings as representations, not magic outputs. They test whether you can explain encoder, metric, index tradeoffs, and practical engineering concerns like cost, versioning, and evaluation. Showing a clear example and awareness of failure modes demonstrates readiness to build reliable retrieval systems.