What are embeddings?

In this question, we will learn what embeddings are, why they help LLM systems, and how to reason about similarity and tradeoffs. We will keep examples concrete so you can explain and implement embeddings in interviews and systems design conversations.

We will cover the following:

• The intuition
• How it actually works
• When to use embeddings versus other representations
• Tradeoffs and failure modes
• Questions the interviewer might ask

Direct answer: Embeddings are dense numeric vectors produced by an encoder that place similar inputs near each other in a $d$-dimensional space, so you can measure similarity with distances or angles. They let you turn text or other data into numbers usable by nearest-neighbor search, clustering, or as inputs to downstream models.

The intuition (an analogy that makes it click)

Think of a map of a city. Each location is a point on the map, and places that are close on the map are close in the real world. Embeddings are like coordinates on that map for words, sentences, or documents. When two texts are about the same topic they land close together, and when they are unrelated they land far apart.

This geometric view makes tasks simple. If you want similar answers to a question, you find nearby points. If you want to cluster customer complaints, you gather points that cluster in the space.

How it actually works (the real mechanics, with one concrete worked example)

An encoder model, which can be a transformer or a simpler network, processes input and outputs a vector $\mathbf{e}\in\mathbb{R}^d$. The components are floats. You then use a similarity function, commonly cosine similarity or Euclidean distance, to compare vectors.

Cosine similarity between two vectors $\mathbf{e}_1$ and $\mathbf{e}_2$ is

Worked example. Suppose a tiny encoder produces two 3-dimensional embeddings:

• $\mathbf{e}_1=[0.1, 0.2, 0.3]$
• $\mathbf{e}_2=[0.0, 0.4, 0.4]$

Compute the dot product and norms:

Then cosine similarity is

A value near 1 means the two embeddings are very similar in direction. In real systems $d$ is often 128, 512, or 1536.

If we compare representations, here is a short table of common types and their characteristics:

When to use embeddings versus other representations

Use embeddings when you care about semantic similarity rather than exact token matches. They shine in retrieval-augmented generation, semantic search, clustering, and recommendation. For strict lexical matches or when you need exact token counts, TF-IDF or one-hot can be simpler and cheaper.

Practical checklist for using embeddings:

• If you need nearest-neighbor search across millions of documents, plan for an ANN index and storage budget.
• If you need interpretability, pair embeddings with explainable features or highlight retrieved contexts.
• If latency is critical, consider lower-dimensional embeddings or quantization techniques.

Tradeoffs and failure modes

Embeddings are powerful but not perfect. They compress meaning into limited dimensions and can lose fine-grained distinctions. They also reflect the data they were trained on, so bias and unwanted correlations appear in the vector space.

Other failure patterns include out-of-distribution inputs mapping unpredictably, adversarial prompts that shift positions, and drift over time as the data distribution changes.

Questions the interviewer might ask

Some follow-up questions you might get:

How do you store and search embeddings at scale? You explain approximate nearest neighbor (ANN) indexes like HNSW or IVF+PQ, memory and disk tradeoffs, and batching for vector similarity queries.

Why use cosine similarity instead of Euclidean distance? Cosine focuses on vector direction and is scale-invariant, which is useful when embedding magnitude is less informative than orientation. Euclidean is sensitive to magnitude and can matter if magnitude carries meaning.

Can you fine-tune embeddings for a specific task? Yes. You can fine-tune or train task-specific encoders with contrastive or supervised loss so that relevant examples are closer in the space.

How do you evaluate embedding quality? Use retrieval metrics like recall@k, NDCG, precision at k, and downstream task performance such as classification or clustering purity.

What are common dimensionalities and how does dimension affect results? Typical sizes are 128, 256, 512, 768, 1024, 1536. Higher dimension can capture nuance but increases storage and compute. Empirically test using validation metrics.

Some things to note:

• Vector indexes add complexity but are essential for sub-second retrieval at scale.
• Always validate on real queries, not just random similarity tests.

What the interviewer is really testing

They want to know you grasp the mapping from discrete data to continuous geometry, how similarity is measured, and the system-level implications like storage, indexing, and evaluation. They also want to see practical awareness of limitations, monitoring needs, and how to choose representations for a concrete use case.