Prompting, RAG or Fine-Tuning?
Spoke & Chain now wants two things: answers about any of its 2,000 parts, whose prices change weekly, and replies that always sound like the shop — warm, brief, a little nerdy about bikes. A consultant suggested "training our own model". That is one option, but usually the most expensive one, and often the wrong one.
The rule
Prompt first, retrieve for facts, fine-tune for behaviour.
There are three main ways to shape what a model produces:
- Prompting — change the input: instructions, context, examples. Instant, cheap, easy to revise. Always the first thing to try.
- Retrieval-augmented generation (RAG) — before each call, search your own documents for the passages relevant to the question and paste them into the prompt. The model then answers from those passages. Ideal for large or frequently changing knowledge.
- Fine-tuning — train an existing model further on many example inputs and outputs, which adjusts its weights. Good for a consistent style, format or narrow skill across huge numbers of calls. It needs good training data, time and money, and must be redone when needs change.
A related, more technical option is prompt tuning: instead of changing the model's weights, a small set of learned numeric vectors is trained and placed in front of every input — a "soft prompt" that no human wrote. It sits between prompting and fine-tuning in cost.
The key point: fine-tuning is poor at teaching fresh facts. Facts baked into weights go stale and can still be recalled wrongly. For facts, retrieval keeps the source current and checkable.
The shortcut
Ask two questions in order:
- Is the problem missing knowledge? → retrieval (or simply pasting the facts, if they are small).
- Is the problem behaviour that good prompts cannot make consistent? → consider fine-tuning.
If neither, keep improving the prompt.
Parts and prices
Missing knowledge that changes weekly. Too big to paste whole. Choose RAG: search the parts list for each question and include the top few matches.
Drills
Answer fast — trust the rule.
Which approach should you almost always try first?
What does RAG add to each call?
Prices change every week. Why is fine-tuning a poor way to teach them?
Which problem is the best fit for fine-tuning?
Remember
- Prompt first, retrieve for facts, fine-tune for behaviour.
- Shortcut: missing knowledge → retrieval; inconsistent behaviour at scale → maybe fine-tune.
- Trap: fine-tuning to teach facts that change.