Chain of Thought and Tree of Thought
Asha asks: "Two sleeper tickets are ₹1,180 each, plus a ₹35 booking fee per ticket. Is that under my ₹2,500 limit?" Answering in one breath, a model might say "Yes, ₹2,360" and forget the fees. Given room to work it through, it gets ₹2,430 — still yes, but now for the right reason. How a model is allowed to think changes what it gets right.
The rule
Let the model think in steps — one line of steps, or several branches compared.
Chain of thought (CoT) asks the model to write out intermediate reasoning before the final answer. Because the model produces one token at a time, the steps it has already written become part of its context, so later steps build on them instead of leaping:
Tickets: 2 × ₹1,180 = ₹2,360
Fees: 2 × ₹35 = ₹70
Total: ₹2,360 + ₹70 = ₹2,430
₹2,430 ≤ ₹2,500 → within budget, ₹70 to spare.You can trigger this with an instruction ("work through the calculation step by step, then give the answer on the last line") or with examples that show worked steps. Some models, often called reasoning models, are trained to do this kind of thinking on their own before answering.
Tree of thought (ToT) goes further for problems with several possible routes. Instead of one chain, it:
- Branches — proposes a few different partial approaches.
- Evaluates — scores each one against the goal.
- Prunes and expands — drops weak branches and develops the promising ones.
It is usually built as a loop in your program, calling the model several times, rather than as a single prompt.
The shortcut
Match the method to the shape of the problem. One path with several steps → chain. Several paths worth comparing → tree. A simple lookup → neither; extra thinking there only adds tokens and delay.
Branch
Three ideas: a direct train Thursday night; Pune to Mumbai, then an overnight train; a bus to Ahmedabad, then a train.
The trap is paying for deep reasoning everywhere. Chains and trees make answers longer, slower and costlier. Use them where a wrong quick answer is likely; skip them for "what is the weather in Jaipur?".
Drills
Answer fast — trust the rule.
Why does chain-of-thought prompting often improve multi-step answers?
Which task best suits tree of thought?
Three tickets cost ₹700 each with a ₹40 fee per ticket. What total should a careful chain reach?
What is the main cost of using chain of thought on every request?
Remember
- Let the model think in steps: a chain for one path, a tree for several paths to compare.
- Shortcut: one path → chain; many paths → tree; a lookup → neither.
- Trap: paying for deep reasoning on simple questions.