Self-Consistency and Ensembling
Chain-of-thought made the refund check far better, but a few borderline cases still flipped: run the same email twice and one run said Refund: yes, the other Refund: no. One reasoning path had simply taken a wrong turn. Instead of trusting a single path, you can ask several times and let the answers vote.
The rule
Ask several times, trust the majority — errors scatter, truth agrees.
Self-consistency works like this:
- Use a chain-of-thought prompt.
- Run it several times — say five — at a temperature above zero, so each run reasons along a slightly different path.
- Ignore the reasoning text and collect only the final answers.
- Pick the answer that appears most often.
Why it helps: there are many ways to go wrong, and they tend to lead to different wrong answers. Correct reasoning paths tend to converge on the same answer. So the majority is more reliable than any single run.
The vote split is useful information too. Five out of five agreeing is a strong signal; three against two says the case is genuinely unclear.
Prompt ensembling is a close cousin. Instead of running one prompt many times, you run several different prompts — different wordings, different example sets — and combine their answers. This reduces the risk that one prompt's particular wording biases every answer the same way.
Run Final line
1 Refund: yes
2 Refund: yes
3 Refund: no
4 Refund: yes
5 Refund: yes
Majority: yes (4 of 5)The shortcut
Think "five runs, one vote, read the margin".
Set up
Chain-of-thought refund prompt, temperature 0.7, five runs of the same email.
The price is plain: five runs cost about five times as much and take longer. Use self-consistency for decisions that matter and have a single checkable answer, like eligibility or a calculation. It does not suit open-ended writing, where there is no final answer to vote on.
Running at temperature 0 defeats the purpose: every run would take nearly the same path and repeat the same mistake.
Drills
Answer fast — trust the rule.
In self-consistency, what do the runs vote on?
Why must the temperature be above zero for self-consistency?
Five runs give: no, yes, no, no, yes. What should the system answer?
How does prompt ensembling differ from self-consistency?
Remember
- Ask several times, trust the majority — errors scatter, truth agrees.
- Shortcut: five runs, one vote, read the margin; send close splits to a human.
- Trap: running at temperature 0, or voting on open-ended writing.