Here’s a homework problem from a class I taught:
… In past years, the average number of accidents per year was 15, and this year it was 10. Is it justified to claim that the accident rate has dropped?
The naive answer is of course the rate has dropped. Ten is less than fifteen. This reminds me of a joke attributed to Abraham Lincoln:
Q: If you call a tail a leg, how many legs does a dog have?
A: Four. Calling a tail a leg doesn’t make it one.
But the homework problem isn’t asking whether ten is less than fifteen. Part of the purpose of the exercise is to state the problem well. The real question is whether there is good evidence that the fundamental causes of automobile accidents have changed for the better or whether there is a fair chance that a random fluctuation caused the decrease. It turns out the latter is the case given the model (Poisson) that the question suggests using.
Think of this example next time you hear politicians say that some measure improved during their administration: economic growth, employment, crime rates, etc. The basic question is whether in fact the measure changed. The next question is whether the change was more likely a coincidence or a genuine improvement. And if there was a real improvement, ask whether the politician deserves any credit.
(The homework exercise came from Statistical Inference, problem 8.2.)