Fine-tuning

How it works

Fine-tuning adapts a pretrained model to a specific task or style by continuing training on a smaller, curated dataset, which updates the model's weights. Where a base model learned general language from a huge corpus, fine-tuning nudges it toward your domain, format, or tone using hundreds to thousands of examples. Parameter-efficient methods like LoRA adjust only a small set of added weights, which makes fine-tuning cheaper and avoids retraining the whole model. The result is a model that behaves the way your examples showed it, without you having to describe that behavior in every prompt.

When it matters

Fine-tuning matters when prompting and retrieval are not enough: you need a consistent output format, a specialized tone, or a task the base model handles unreliably. It is often the wrong first reach, though. For adding knowledge or citable facts, RAG is usually cheaper and more current, because fine-tuning bakes information into weights you then have to retrain to update.

Common misconceptions

• "Fine-tuning teaches the model new facts." It mostly teaches behavior and style; use RAG for fresh or proprietary knowledge.
• "It is the first thing to try." Prompting and retrieval come first; fine-tune when they plateau.
• "Fine-tuning equals training from scratch." It continues from a pretrained model, which is far cheaper.