It’s a simple rule of probability that if A makes B more likely, B makes A more likely. That is, if the conditional probability of A given B is larger than the probability of A alone, the the conditional probability…
Blog Archives
Closet Bayesian
When I was a grad student, a statistics postdoc confided to me that he was a “closet Bayesian.” This sounded absolutely bizarre. Why would someone be secretive about his preferred approach to statistics? I could not imagine someone whispering that…
Sleeper theorems
I’m using the term “sleeper” here for a theorem that is far more important than it seems, something that you may not appreciate for years after you first see it. The first such theorem that comes to mind is Bayes…
Product of normal PDFs
The product of two normal PDFs is proportional to a normal PDF. This is well known in Bayesian statistics because a normal likelihood times a normal prior gives a normal posterior. But because Bayesian applications don’t usually need to know…
Shifting probability distributions
One reason the normal distribution is easy to work with is that you can vary the mean and variance independently. With other distribution families, the mean and variance may be linked in some nonlinear way. I was looking for a…
Fast approximation of beta inequalities
A beta distribution has an approximate normal shape if its parameters are large, and so you could use normal approximations to compute beta inequalities. The corresponding normal inequalities can be computed in closed form. This works surprisingly well. Even when…
How do you justify that distribution?
Someone asked me yesterday how people justify probability distribution assumptions. Sometimes the most mystifying assumption is the first one: “Assume X is normally distributed …” Here are a few answers. Sometimes distribution assumptions are not justified. Sometimes distributions can be…
Tagged with: Bayesian, Clinical trials, Probability and Statistics
Posted in Clinical trials, Statistics
Posted in Clinical trials, Statistics
Vague priors are informative
Data analysis has to start from some set of assumptions. Bayesian prior distributions drive some people crazy because they make assumptions explicit that people prefer to leave implicit. But there’s no escaping the need to make some sort of prior…
Avoiding underflow in Bayesian computations
Here’s a common problem that arises in Bayesian computation. Everything works just fine until you have more data than you’ve seen before. Then suddenly you start getting infinite, NaN, or otherwise strange results. This post explains what might be wrong…
Monkeying with Bayes' theorem
In Peter Norvig’s talk The Unreasonable Effectiveness of Data, starting at 37:42, he describes a translation algorithm based on Bayes’ theorem. Pick the English word that has the highest posterior probability as the translation. No surprise here. Then at 38:16…
The universal solvent of statistics
Andrew Gelman just posted an interesting article on the philosophy of Bayesian statistics. Here’s my favorite passage. This reminds me of a standard question that Don Rubin … asks in virtually any situation: “What would you do if you had…
Six analysis and probability diagrams
Here are a few diagrams I’ve created that summarize relationships in analysis and probability. Click on a thumbnail image to go to a page with the full image and explanatory text. Special functions Gamma and related functions Probability distributions Conjugate…
Interpreting statistics
From Matt Briggs: I challenge you to find me in any published statistical analysis, outside of an introductory textbook, a confidence interval given the correct interpretation. If you can find even one instance where the [frequentist] confidence interval is not…
Bad logic, but good statistics
Ad hominem arguments are bad logic, but good (Bayesian) statistics. A statement isn’t necessarily false because it comes from an unreliable source, though it is more likely to be false. Some people are much more likely to know what they’re…
Leading digits of factorials
Suppose you take factorials of a lot of numbers and look at the leading digit of each result. You could argue that there’s no apparent reason that any digit would be more common than any other, so you’d expect each…
