Chain-of-Thought Prompting
"Is this customer owed a refund?" sounds like a yes-or-no question, but answering it means finding the delivery date, counting days, checking whether the item was used and comparing with the policy. When the prompt demanded only "yes" or "no", the model got about one in four wrong. Letting it work the problem through first changed that.
The rule
Reason first, answer last — the answer can only use what came before.
Chain-of-thought (CoT) prompting asks the model to write out intermediate reasoning steps before its final answer. It helps on tasks with several steps: arithmetic, date logic, rule-checking, planning.
Why it works follows from next-token prediction. Each token is chosen based on the text before it. If the final answer comes first, it must be produced in a single step, with no written working to lean on. If the reasoning comes first, the answer is predicted after the dates, counts and comparisons are already written down in the context.
Two common forms:
- Zero-shot CoT — add an instruction like "Work through this step by step before answering."
- Few-shot CoT — include examples whose outputs show the reasoning, not just the answer.
Policy: unused items returned within 30 days of delivery get a full refund.
Today is 20 June.
<email>The gloves arrived on 28 May. Never worn. Can I return them?</email>
Reason step by step: find the delivery date, count the days to today,
check the item's condition, compare with the policy.
Then write a final line: "Refund: yes" or "Refund: no".A good response counts 3 days left in May plus 20 in June, 23 days in total, notes "never worn", and ends Refund: yes.
The shortcut
Use CoT when a careful human would need scratch paper. Then fix the answer's position and format so a program can find it.
Needs scratch paper?
Dates, counting and a rule check: yes. One-word triage labels: usually no.
Costs to remember: reasoning adds tokens, so it is slower and pricier; and the written reasoning is not a guaranteed faithful record of how the answer was produced. Judge CoT by whether final answers improve on your test set. Many newer models reason internally by default; for them, a clear request for careful checking plus a fixed answer format is often enough.
Drills
Answer fast — trust the rule.
Why should the reasoning come before the final answer?
Which task benefits most from chain-of-thought?
The refund email says delivered 25 May; today is 20 June; policy is 30 days. How many days have passed?
What is a real drawback of chain-of-thought?
Remember
- Reason first, answer last — the answer can only use what came before.
- Shortcut: use CoT when a human would need scratch paper; fix the final line's format.
- Trap: paying for reasoning on simple tasks, or trusting the reasoning text without testing the answers.