Pebellslearn by building
0%
RoadmapsMatrixCoursesSandboxSign in
Statistical Inference  ›  Lesson

Samples and Standard Error

Statistics 14 minA statistic computed from a sample varies from sample to sample, and the standard error measures that variation, shrinking with the square root of the sample size.

Samples and Standard Error

Pantry has 50,000 customers. It surveys 400 of them, chosen at random, and the average delivery rating comes back as 4.1 out of 5. Nobody asked the other 49,600. How far from 4.1 could the true average be? If you cannot answer that, you cannot say whether next month's 4.0 is a real drop or just a different 400 people.

The rule

Every estimate wobbles; the standard error says by how much.

The population is everyone you care about: all 50,000 customers. The sample is the 400 you measured. Draw a different random 400 and the average would come out a little different, perhaps 4.07 or 4.13. The standard error (SE) is the typical size of that sample-to-sample wobble.

For an average, divide the spread of individual values by the square root of the sample size:

SEmean=snSE_{\text{mean}} = \frac{s}{\sqrt{n}}

Ratings have a standard deviation s=0.8s = 0.8, and 400=20\sqrt{400} = 20, so SE=0.8/20=0.04SE = 0.8 / 20 = 0.04. The survey average is typically within a few hundredths of the truth.

For a proportion, such as the share of users who convert, the spread comes from the proportion itself:

SEprop=p(1−p)nSE_{\text{prop}} = \sqrt{\frac{p(1-p)}{n}}

Keep two ideas apart. The standard deviation describes how much individual customers differ, and it does not shrink as you ask more people. The standard error describes how much your average would move between samples, and it does shrink.

The shortcut

Because nn sits under a square root, four times the sample buys half the wobble. A hundred times the sample buys a tenth. Precision is expensive, so decide how much you need before you collect.

SE of Quick Checkout's control groupStep 1 of 4

  1. Write down p and n

    In the checkout test, 14,400 users saw the old checkout and 10% of them ordered. So p = 0.1 and n = 14,400.

The standard error only measures random wobble. If the survey went only to customers who opened a "rate your delivery" email, a larger sample of the same people is still tilted towards the happy ones. Size cures noise, not bias.

Drills

Answer fast — trust the rule.

Ratings have a standard deviation of 0.8. With 400 random responses, what is the standard error of the average?
0.8÷400= ?0.8 \div \sqrt{400} = \,?
quick drill

How many responses would cut that standard error from 0.04 to 0.02?

20% of 2,500 users reorder within a week. What is the standard error of that proportion?
0.2×0.82500= ?\sqrt{0.2 \times \tfrac{0.8}{2500}} = \,?
quick drill

As Pantry collects more survey responses, which number shrinks?

quick drill

The survey only reaches customers who opened a "rate us" email, who tend to be the happiest. What would collecting 4,000 such replies instead of 400 do?

Remember

  • Every estimate wobbles; the standard error measures the wobble.
  • SE = spread ÷ √n, so four times the sample halves it.
  • Trap: a big sample does not fix a biased one.

Practice — level up

0/1 solved

Write your answer, then compare it with a rubric and a model answer.

ExplainIs 400 people enough?

A manager says: 'We have 50,000 customers. An average from 400 of them is meaningless.' The ratings have a standard deviation of 0.8. Explain why a random sample of 400 can still give a useful average rating, how precise it is, and what the standard error does not protect against.