Correlation, Confounding and Causation
Pantry's app has a recipes tab. Customers who use it order 30% more groceries than those who do not. The marketing team wants to push recipe notifications to everyone, expecting a 30% jump in orders. Before anyone spends money, a data scientist asks one question: is the recipes tab causing those extra orders, or just keeping company with them?
The rule
Before trusting a link, ask what else could drive both.
A correlation says two things move together. A causal effect says that changing one would change the other. The gap between them is filled by rival explanations:
- Confounding: a third factor drives both. People who love cooking open recipes and buy more ingredients, and they would buy more even if the tab vanished. That third factor, cooking enthusiasm, is a confounder.
- Reverse causation: the arrow points the other way. Heavy shoppers open the app more often, so they stumble onto the recipes tab more often.
- Selection: the groups were formed in a way that already made them different. The tab only exists in the newest app version, and loyal customers update first.
- Chance: with small groups, a pattern can appear by luck.
Each rival predicts the same correlation you saw. That is why the correlation alone cannot tell you what pushing recipes to everyone would do. If most of the 30% gap is cooking enthusiasm, the notifications will add almost nothing.
The shortcut
For any "X goes with Y" claim, run the four-rival check out loud: reverse, confounder, selection, chance. If any rival is plausible, the claim needs a design that rules it out, and the strongest such design is a randomised experiment, the subject of the next lesson.
Reverse
Could ordering more cause recipe use? Yes: frequent shoppers open the app more and find the tab more.
The trap is thinking more data fixes this. A million observational rows reproduce the same confounded gap with more confidence, not less bias.
Drills
Answer fast — trust the rule.
Members who contact support cancel more often. A manager concludes that support drives customers away. What is the most likely confounder?
Customers who open the app daily spend more. Which rival explanation is reverse causation?
What must be true of a confounder?
Which approach rules out confounders you never measured?
Remember
- Before trusting a link, ask what else could drive both.
- Four-rival check: reverse, confounder, selection, chance.
- Trap: believing more observational data removes confounding. It only makes the biased answer look surer.