The law of small numbers says that people underestimate the variability in small samples. Said another way, people overestimate what can be accomplished with a small study. Here’s a simple example. Suppose a drug is effective in 80% of patients. If five patients are treated, how many will respond?
Many people reason that 80% means 4 out of 5, so if 5 people are treated, exactly 4 will respond. Always.
Others understand that things are not guaranteed to work out so neatly, but they still believe that it is highly likely that 4 people would respond. Maybe a 90% chance.
In fact, there’s only a 41% chance that exactly 4 would respond out of a sample of 5.
Dear,
I liked a lot of the post. However, would You mind to show us the calculations you did to get this result of 41%?
Thanks in advance
and Congratulaions for the Blog. You are in my RSS, even though I am a political science graduate student in Brazil.
Manoel,
Thanks for your note. Here’s where the 41% comes from. First think about the probability of four responses followed by one failure: 0.84 * 0.2. But this is only one possibility. The failure could be the first patient, or the second, etc. for a total of five possibilities. So we need to multiply the preliminary probability by 5. The result is this 5 * 0.84 * 0.2 = 0.4096, which I rounded up to 0.41.
Ah, I get it. Why I didn’t think that before? Maybe because my statistic teacher didn’t stressed the probabilities part of the course. Or, either, because I was worried about doing the calculations right to go well in the test, but didn’t make the connections between things like this.
Anyway,
Thanks!