The law of small numbers says that people underestimate the variability in small samples. Said another way, people overestimate what can be accomplished with a small study. Here’s a simple example. Suppose a drug is effective in 80% of patients. If five patients are treated, how many will respond?

Many people reason that 80% means 4 out of 5, so if 5 people are treated, exactly 4 will respond. Always.

Others understand that things are not *guaranteed* to work out so neatly, but they still believe that it is highly likely that 4 people would respond. Maybe a 90% chance.

In fact, there’s only a 41% chance that exactly 4 would respond out of a sample of 5.

Dear,

I liked a lot of the post. However, would You mind to show us the calculations you did to get this result of 41%?

Thanks in advance

and Congratulaions for the Blog. You are in my RSS, even though I am a political science graduate student in Brazil.

Manoel,

Thanks for your note. Here’s where the 41% comes from. First think about the probability of four responses followed by one failure: 0.8

^{4}* 0.2. But this is only one possibility. The failure could be the first patient, or the second, etc. for a total of five possibilities. So we need to multiply the preliminary probability by 5. The result is this 5 * 0.8^{4}* 0.2 = 0.4096, which I rounded up to 0.41.Ah, I get it. Why I didn’t think that before? Maybe because my statistic teacher didn’t stressed the probabilities part of the course. Or, either, because I was worried about doing the calculations right to go well in the test, but didn’t make the connections between things like this.

Anyway,

Thanks!

Very good example and statistical explanation! Cheers!