What is a confidence interval?

At first glance, a confidence interval is simple. If we say [3, 4] is a 95% confidence interval for a parameter θ, then there’s a 95% chance that θ is between 3 and 4. That explanation is not correct, but it works better in practice than in theory.

If you’re a Bayesian, the explanation above is correct if you change the terminology from “confidence” interval to “credible” interval. But if you’re a frequentist, you can’t make probability statements about parameters.

Confidence intervals take some delicate explanation. I took a look at Andrew Gelman and Deborah Nolan’s book Teaching Statistics: a bag of tricks,  to see what they had to say about teaching confidence intervals. They begin their section on the topic by saying “Confidence intervals are complicated …” That made me feel better. Some folks with more experience teaching statistics also find this challenging to teach. And according to The Lady Testing Tea, confidence intervals were controversial when they were first introduced.

From a frequentist perspective, confidence intervals are random, parameters are not, exactly the opposite of what everyone naturally thinks. You can’t talk about the probability that θ is in an interval because θ isn’t random. But in that case, what good is a confidence interval? As L. J. Savage once said,

The only use I know for a confidence interval is to have confidence in it.

In practice, people don’t go too wrong using the popular but technically incorrect notion of a confidence interval. Frequentist confidence intervals often approximate Bayesian credibility intervals; the frequentist approach is more useful in practice than in theory.

It’s interesting to see a sort of détente between frequentist and Bayesian statisticians. Some frequentists say that the Bayesian interpretation of statistics is nonsense, but the methods these crazy Bayesians come up with often have good frequentist properties. And some Bayesians say that frequentist methods, such as confidence intervals, are useful because they can come up with results that often approximate Bayesian results.

 

Click to learn more about Bayesian statistics consulting

 

7 thoughts on “What is a confidence interval?

  1. Does it mean that : [3,4] has 95% of chance to contain the real value of the paremeter ?

    I don’t understand the approximation of baysian interval by frequentist one.

    Last point : thank you very much for your blog.

  2. Julien: The interpretation you give for a confidence interval is the one nearly everyone believes but isn’t correct. The precise definition is complicated and unintuitive.

  3. Hi John,

    I still don’t quite understand. Let’s say I have a test statistic and build an empirical 95% confidence interval. Is it true to say I have 95% confidence my interval contains the true statistic?

Leave a Reply

Your email address will not be published. Required fields are marked *