Definition

Fine-tuning is the process of taking a model that has already been pretrained and training it further on a smaller, more targeted dataset so it adapts to a specific task, domain or style.

Pretraining gives a model general language or vision skills, but it may not know your industry’s terminology or produce output in the format you need. Fine-tuning is like on-the-job training for a generalist: the core abilities stay, and performance improves on a particular kind of work. It needs far less data and compute than training from scratch.

How It Works

The basic workflow:

  1. Prepare data: shape it into learnable examples, such as instruction/ideal-response pairs or input/label pairs. Quality usually matters more than quantity.
  2. Pick a base model: choose a pretrained model based on task difficulty, deployment constraints and license.
  3. Train: update parameters on the new data with a small learning rate, nudging the model toward the target behavior.
  4. Evaluate: test on held-out data, and confirm general abilities haven’t noticeably degraded.

By how many parameters get updated, fine-tuning falls into two broad groups:

Full fine-tuning. All parameters are updated. The ceiling is high, but it needs a lot of GPU memory, and each task requires saving a full copy of the model.

Parameter-efficient fine-tuning (PEFT). Most of the original model is frozen and only a small set of new or selected parameters is trained. The best-known method, LoRA (low-rank adaptation), adds two small matrices alongside some of the model’s weights and trains only those. Memory needs drop sharply, the output is small, and you can keep separate “adapters” for different tasks and load them as needed. QLoRA combines this with quantization so larger models can be fine-tuned on modest hardware.

By purpose, common types include:

  • Instruction tuning: teaching the model to understand and follow a wide range of instructions.
  • Domain tuning: familiarizing it with the language and conventions of fields like medicine, law or finance.
  • Task tuning: optimizing for classification, extraction or a specific output format.

There are risks. With too little or too narrow data, models overfit. Training too hard on new data can erode existing general abilities, a problem often called catastrophic forgetting.

Typical Applications

  • Getting a support model to use a company’s standard phrasing and tone
  • Making a model reliably produce a specific JSON structure
  • Improving comprehension and writing on specialized domain text
  • Training a small model for one task to cut deployment costs
  • Teaching an image model a particular art style or character

If the goal is knowledge that changes often, retrieval-augmented generation (RAG) is usually the better fit. Fine-tuning is better at changing how a model behaves.

  • Pretraining: the starting point that gives the model general abilities.
  • RLHF: further training driven by human preferences.
  • RAG: adds knowledge through retrieval; often compared or combined with fine-tuning.
  • Quantization: lowers precision to save resources; QLoRA pairs it with fine-tuning.
  • Large language model: one of the most common targets for fine-tuning.

Paper: LoRA: Low-Rank Adaptation of Large Language Models