At first glance, a confidence interval is simple. If we say [3, 4] is a 95% confidence interval for a parameter θ, then there’s a 95% chance that θ is between 3 and 4. That explanation is not correct, but it works better in practice than in theory.
If you’re a Bayesian, the explanation above is correct if you change the terminology from “confidence” interval to “credible” interval. But if you’re a frequentist, you can’t make probability statements about parameters.
Confidence intervals take some delicate explanation. I took a look at Andrew Gelman and Deborah Nolan’s book Teaching Statistics: a bag of tricks, to see what they had to say about teaching confidence intervals. They begin their section on the topic by saying “Confidence intervals are complicated …” That made me feel better. Some folks with more experience teaching statistics also find this challenging to teach. And according to The Lady Testing Tea, confidence intervals were controversial when they were first introduced.
From a frequentist perspective, confidence intervals are random, parameters are not, exactly the opposite of what everyone naturally thinks. You can’t talk about the probability that θ is in an interval because θ isn’t random. But in that case, what good is a confidence interval? As L. J. Savage once said,
The only use I know for a confidence interval is to have confidence in it.
In practice, people don’t go too wrong using the popular but technically incorrect notion of a confidence interval. Frequentist confidence intervals often approximate Bayesian credibility intervals; the frequentist approach is more useful in practice than in theory.
It’s interesting to see a sort of détente between frequentist and Bayesian statisticians. Some frequentists say that the Bayesian interpretation of statistics is nonsense, but the methods these crazy Bayesians come up with often have good frequentist properties. And some Bayesians say that frequentist methods, such as confidence intervals, are useful because they can come up with results that often approximate Bayesian results.
Does it mean that : [3,4] has 95% of chance to contain the real value of the paremeter ?
I don’t understand the approximation of baysian interval by frequentist one.
Last point : thank you very much for your blog.
Julien: The interpretation you give for a confidence interval is the one nearly everyone believes but isn’t correct. The precise definition is complicated and unintuitive.
Hi John,
I still don’t quite understand. Let’s say I have a test statistic and build an empirical 95% confidence interval. Is it true to say I have 95% confidence my interval contains the true statistic?
Alex; it is true to say that in repeated sampling your confidence interval will contain the true parameter (what we are interested in) 95% of the time.
(Under frequentist paradigm) confidence intervals change between samples, but the parameter is a fixed point i.e. it is a category error to talk about its probability. We can express probabilities regarding the confidence intervals but not the parameter. Under the frequentist paradigm, all the probability statements express the frequencies of results of a given process in repeated trials.
So I guess nothing really stops you from informally expressing this situation as “95% confidence my interval contains the true statistic” and nothing will noticeably go wrong if you live as if that is the case but strictly speaking it is not a deduction you can justify.