Andrew Gelman on subjectivity in statistics:
Bayesian methods are often characterized as “subjective” because the user must choose a prior distribution, that is, a mathematical expression of prior information. The prior distribution requires information and user input, that’s for sure, but I don’t see this as being any more “subjective” than other aspects of a statistical procedure, such as the choice of model for the data (for example, logistic regression) or the choice of which variables to include in a prediction, the choice of which coefficients should vary over time or across situations, the choice of statistical test, and so forth. Indeed, Bayesian methods can in many ways be more “objective” than conventional approaches in that Bayesian inference, with its smoothing and partial pooling, is well adapted to including diverse sources of information and thus can reduce the number of data coding or data exclusion choice points in an analysis.
People worry about prior distributions, not because they’re subjective, but because they’re explicitly subjective. There are many other subjective factors, common to Bayesian and Frequentist statistics, but these are implicitly subjective.
In practice, prior distributions often don’t make much difference. For example, you might show that an optimistic prior and a pessimistic prior lead to the same conclusion.
If you have so little data that the choice of prior does make a substantial difference, being able to specify the prior is a benefit. Suppose you only have a little data but have to make a decision anyway. A frequentist might say there’s too little data for a statistical analysis to be meaningful. So what do you do? Make a decision entirely subjectively! But with a Bayesian approach, you capture what is known outside of the data at hand in the form of a prior distribution, and update the prior with the little data you have. In this scenario, a Bayesian analysis is less subjective and more informed by the data than a frequentist approach.
Well said!
I wish I had used similar words 30 years ago when I was trying to explain the need to use Kalman filtering on a “novel” radiation sensor that needed both fast and accurate response. Other smoothing/filtering algorithms were better understood, with a long history in the notoriously conservative nuclear industry.
Despite having a ton of solid lab data to support my argument, my poor conceptual explanation of *why* the updating of priors in the Kalman filter was both correct and optimal caused us not to use it.
I understood the theory (well enough to write and verify code), but was unable to communicate it effectively.
“There are many other subjective factors, common to Bayesian and Frequentist statistics, but these are implicitly subjective.”
This is why it always makes me laugh when people tell me that they live their lives by “facts”, while haughtily decrying the beliefs of others (Neil deGrasse Tyson comes to mind). The beliefs these people have are built on “facts” that are derived in sujective manners, such as “the choice of model for the data (for example, logistic regression) or the choice of which variables to include in a prediction, the choice of which coefficients should vary over time or across situations, the choice of statistical test”.
Literally every “fact” is built on a prior belief that has no logical foundation than an axiom.
I no longer try to explain the above to them because they adamantly insist their choice of models is built on other “facts”. It’s turtles all the way down.
Ken: Along those lines, there are people who say they take nothing on authority. Why believe relativity? Have you ever experienced time dilation, for example? No, we trust the authorities who tell us that experiments confirm relativity, and reasonably so.
Information and data points do not share a proportional correspondence with one another, in general. It is, therefore, inappropriate to forge ahead with insufficient data and say it’s better than nothing. It’s often worse.