Correlation and Covariance
The owner wonders: "When a book is cheaper, do people buy more copies of it?" That is a question about two columns moving together. Covariance and correlation turn "moving together" into a number, and correlation puts that number on a scale anyone can read.
The rule
Correlation runs from −1 to +1: sign gives direction, size gives strength.
Covariance checks whether two columns sit above their means at the same time. Positive: they tend to rise together. Negative: one rises as the other falls.
sales[["copies", "price"]].cov()
# copies price
# copies 2.79 -22.50
# price -22.50 13826.79The sign is useful, but the size is not: it depends on units. Measure price in paise instead of rupees and the covariance becomes 100 times bigger, while the relationship is unchanged.
Correlation (Pearson's r) fixes that by dividing covariance by both standard deviations. The result always lands between −1 and +1:
- +1: a perfect rising straight line.
- −1: a perfect falling straight line.
- 0: no straight-line relationship.
sales["copies"].corr(sales["price"]) # about −0.11
sales[["copies", "price", "revenue"]].corr().round(2)| copies | price | revenue | |
|---|---|---|---|
| copies | 1.00 | −0.11 | 0.59 |
| price | −0.11 | 1.00 | 0.67 |
| revenue | 0.59 | 0.67 | 1.00 |
The diagonal is always 1 (every column matches itself perfectly), and the table is a mirror image across it. Cheaper books show only a very weak link to more copies here: −0.11 is close to 0. Revenue correlates with both, which makes sense, since it is built from them.
The shortcut
Read any r in two glances: sign, then size. Sign: rising together or opposite. Size: under about 0.3 is weak, 0.3 to 0.7 moderate, above 0.7 strong. Then ask the third question every time: could something else explain it?
Sign
Positive: pricier sales tend to bring in more revenue.
The trap is reading correlation as cause. Umbrella sales and traffic accidents rise together, because rain drives both. Also, r only measures straight-line patterns: a perfect U-shape can have r near 0.
Drills
Answer fast — trust the rule.
Which correlation shows the strongest relationship?
What is always on the diagonal of a correlation matrix?
What is pd.Series([1, 2, 3, 4]).corr(pd.Series([8, 6, 4, 2]))?
Why prefer correlation over covariance when comparing relationships?
Remember
- Correlation runs from −1 to +1: sign gives direction, size gives strength.
- Covariance gives direction only; its size depends on units.
- Correlation is not cause, and it only sees straight-line patterns.