What is fine-tuning, and when should you fine-tune an LLM?

In this question, we will learn what fine-tuning means, why and when you would do it, and how to pick a method that fits your data, compute, and deployment constraints. We will emphasize practical signals you can watch for in an interview or a real project.

We will cover the following:

• The intuition
• How it actually works
• Parameter-efficient alternatives
• When to fine-tune versus prompt engineering
• Tradeoffs and failure modes
• Questions the interviewer might ask

Fine-tuning is the process of adapting a pretrained large language model to a target task or domain by updating model weights or adding small trainable components; choose it when your task requires improved accuracy, consistent behavior, or domain-specific knowledge that prompt engineering cannot reliably provide. Fine-tune fully when you have enough labeled data and compute and need maximum performance, or choose parameter-efficient approaches when resources are limited or you want faster iteration.

The intuition (an analogy that makes it click)

Think of a pretrained LLM as a general-purpose chef who learned many cuisines. Prompting is like giving the chef a recipe and a description of the meal you want. Fine-tuning is like sending the chef back to a short apprenticeship focused on your cuisine, so they learn the precise spices and techniques you prefer. The apprenticeship costs time and resources, but gives predictable, reproducible results when repeated.

How it actually works (the real mechanics, with one concrete worked example)

At a high level you pick a pretrained model, assemble labeled or weakly labeled data, choose which parameters to update, and run optimization until the model fits the new distribution. The main axes are dataset size, number of trainable parameters, and compute budget.

A simple worked example. Suppose you have a 7 billion parameter model, and you want to adapt it to classify medical notes with 20k labeled examples. You might choose a parameter-efficient method such as adapters or LoRA to avoid updating all $7\times10^{9}$ parameters.

If $P$ is the total number of parameters and $p$ is the number of trainable parameters, the trainable fraction is

Common choices look like this:

In our 7B example, using LoRA with $p=70$M parameters gives $p/P=0.01$, a 1 percent trainable fraction. Training updates are faster and use less memory, and you can store multiple LoRA modules for different tasks.

Parameter-efficient alternatives

Parameter-efficient fine-tuning (PEFT) methods include adapter modules, LoRA, prefix tuning, and prompt tuning. They let you keep the base model fixed and train small additional parameters. Benefits include lower compute, smaller checkpoint sizes, and easier rollback to the base model.

Performance typically orders as: full-parameter > PEFT > prompt-only, but the gaps shrink as dataset size increases. PEFT often reaches near-full performance on domain adaptation with hundreds to thousands of examples.

When comparing approaches consider three metrics: final accuracy, training cost, and maintenance complexity. Use a small evaluation split to estimate marginal gain per GPU-hour before committing to full fine-tuning.

When to fine-tune versus prompt engineering

Prompt engineering or few-shot prompting is quick and cheap. Use prompting if you need a fast proof of concept, have highly variable tasks, or cannot store custom models. Choose fine-tuning when one or more of the following hold:

• The model repeatedly makes the same kind of mistake that prompts cannot reliably fix.
• You need deterministic, reproducible behavior across many calls and users.
• You need to enforce domain-specific constraints or outputs not covered by general pretraining.
• You have sufficient labeled data or can create high-quality weak labels.

Prompting and fine-tuning are not exclusive. A common workflow is: prototype with prompts, collect failure cases, and then fine-tune a PEFT module on curated examples.

Tradeoffs and failure modes

Fine-tuning increases accuracy at the cost of compute, maintenance, and potential risks. Common failure modes include overfitting to a narrow dataset, catastrophic forgetting of general knowledge, and unintentionally amplifying biases present in fine-tuning data.

Other practical tradeoffs:

• Storage: multiple tuned checkpoints increase storage and deployment complexity.
• Compliance: specialized models may need separate auditing and monitoring.
• Latency and footprint: PEFT can keep latency low by only adding light modules.

Questions the interviewer might ask

Some follow-up questions you might get:

How much data do you need to fine-tune effectively? It depends on task complexity. For narrow classification, hundreds to a few thousand high-quality labels can suffice; for generative domain adaptation, tens of thousands are often better. Evaluate learning curves to see marginal gains.

When would you pick full fine-tuning over LoRA or adapters? Pick full fine-tuning when you have abundant data, plenty of compute, and you need the absolute best possible performance or low-probability behavior that small heads cannot capture.

How do you prevent catastrophic forgetting? Use techniques like replay of a subset of pretraining data, regularization, or multi-task training that mixes original and new data. PEFT methods inherently reduce forgetting by keeping base weights fixed.

Can you combine fine-tuning with prompt engineering? Yes. A typical flow is to prompt to prototype, then fine-tune on curated prompt-response pairs, and finally use prompts to steer behavior at inference time.

How do you evaluate whether fine-tuning is worth it? Measure improvement on held-out, realistic test sets and compute cost per percent improvement or per absolute accuracy gain. Consider maintenance and regulatory cost too.

Some things to note:

• Always track a small validation split that mimics production usage.
• Keep a stable base model checkpoint to rollback when needed.

What the interviewer is really testing

They want to know you can balance technical methods with product and engineering constraints. Good answers show you understand different fine-tuning methods, can read signals when accuracy or control justifies the cost, and can propose safeguards like validation, rollback, and PEFT as a middle ground.