Pebellslearn by building
0%
RoadmapsMatrixCoursesSandboxSign in
Experiments and Causal Effects  ›  Capstone Project

Difference-in-Differences

Econometrics 14 minDifference-in-differences estimates an effect by subtracting a comparison group's change over time from the treated group's change, which removes shared trends as long as both groups would otherwise have moved in parallel.

Difference-in-Differences

In March, Pantry launched 20-minute express delivery in Pune but not in Nashik. By April, Pune's orders were up sharply, and the launch team wants credit for all of it. But March also brought Holi and the start of wedding season, and orders rose in Nashik too, where nothing launched. How much of Pune's rise did express delivery actually cause?

The rule

Subtract the comparison group's change to remove the shared trend.

Neither simple comparison works on its own:

  • Pune before versus after mixes the launch with everything else that changed over those months: season, marketing, prices.
  • Pune versus Nashik after the launch mixes the launch with the gaps that already existed between the two cities.

Difference-in-differences (DiD) subtracts both problems away. First take each city's change over time, then take the difference between those changes:

DiD=(treatedafter−treatedbefore)−(controlafter−controlbefore)\text{DiD} = (\text{treated}_{\text{after}} - \text{treated}_{\text{before}}) - (\text{control}_{\text{after}} - \text{control}_{\text{before}})

Weekly orders per 1,000 customers:

City February April Change
Pune, with express 40 52 +12
Nashik, without 30 36 +6

Nashik's +6 shows what happened anyway. Pune's extra rise beyond that, 12 − 6 = +6 orders per 1,000 customers a week, is the estimated effect of express delivery.

The whole method rests on one assumption, parallel trends: without the launch, Pune would have changed by the same amount as Nashik. The levels may differ (40 against 30); the trends must match. You cannot see the Pune that never got express, but you can check the months before the launch. If both cities rose in step from October to February, the assumption is believable. If Pune was already pulling away, the estimate will credit that head start to express delivery.

The shortcut

Two subtractions: across time within each group, then across groups. Then look at the pre-launch months before believing the answer.

Estimating the express delivery effectStep 1 of 4

  1. Change in the treated city

    Pune: 52 − 40 = +12 orders per 1,000 customers a week.

Drills

Answer fast — trust the rule.

A treated city goes from 50 to 58 weekly orders per 1,000 customers; a comparison city goes from 45 to 49. What is the DiD estimate?
(58−50)−(49−45)= ?(58 - 50) - (49 - 45) = \,?
quick drill

Which assumption does difference-in-differences depend on?

quick drill

Suppose that before the launch, Pune's orders had been rising 3 per 1,000 customers a month faster than Nashik's. What does this do to the +6 estimate?

quick drill

Why not just compare Pune in April (52) with Nashik in April (36)?

Remember

  • Subtract the comparison group's change to remove the shared trend.
  • Two subtractions: over time within each group, then between groups.
  • Trap: trusting DiD without checking that the trends were parallel before the change.

Practice — level up

0/1 solved

Write your answer, then compare it with a rubric and a model answer.

ExplainSix, not twelve

The launch team says express delivery added 12 weekly orders per 1,000 customers in Pune. Explain to the finance team why your estimate is 6 instead, how you calculated it, and what evidence would make you doubt the 6.