Fine Tuning
What is it?
Fine tuning is further training an existing model on your own examples so it specializes in a specific task, tone, or domain.
Explain like I'm 5
Why was it created?
Training from scratch is hugely expensive. Fine tuning was adopted to adapt powerful pre-trained models to specific needs at a fraction of the cost.
Where is it used?
- Specializing models to a domain
- Enforcing a consistent style or format
- Teaching task-specific behavior
- Improving accuracy on narrow tasks
Why should developers care?
It's one of the main ways teams specialize AI, so understanding when it helps (versus prompting or RAG) is a key build decision.
How does it work?
You start from a pre-trained model and continue training it on a curated set of examples for your task. This nudges its parameters toward your desired behavior without learning language from scratch.
Real-world example
A company fine-tunes a model on thousands of its past support replies so it answers in the company's voice and follows its policies.
Common use cases
- Domain-specific assistants
- Consistent tone/format
- Specialized classification
- Reducing prompt complexity for repeated tasks
Advantages
- Cheaper than training from scratch
- Specializes behavior reliably
- Can shrink prompts for repeated tasks
- Improves narrow-task accuracy
Disadvantages
- Needs quality labeled data
- Costs more than prompting/RAG
- Can become stale as needs change
- Risk of overfitting to examples
When should you use it?
When you need consistent, specialized behavior that prompting and RAG can't reliably achieve.
When should you avoid it?
When prompt engineering or RAG already works — they're cheaper and faster to iterate.
Alternatives
Related terms
Interview questions
Beginner
- What is fine tuning?
- How does it differ from training from scratch?
Intermediate
- When would you fine-tune versus use RAG?
- What data do you need to fine-tune?
Senior
- What are the risks of overfitting during fine-tuning?
- How do you decide fine-tuning is worth the cost?
Common misconceptions
- "Fine tuning teaches the model new facts reliably" — it shapes behavior/style more than it's a dependable way to add knowledge; RAG is better for facts.
- "Fine tuning is always better than prompting" — it's costlier and often unnecessary.
Fun facts
- Efficient methods can fine-tune by adjusting only a small slice of a model's parameters.
- For adding up-to-date knowledge, RAG is usually preferred over fine-tuning.
Timeline
- 2020s — Fine-tuning of large pre-trained models becomes common practice
Learning resources
Quick summary
Fine tuning adapts a pre-trained model to your task or style with extra training on your examples — cheaper than scratch, but pricier than prompting or RAG.
Cheat sheet
- Extra training on your examples
- Specializes tone/task/domain
- Best for behavior, not fresh facts
- Try prompting/RAG first