Comments on: Closet Bayesian

By: There is no Theorem but Bayes’ and Laplace is His Prophet | Alea Deum

There is no Theorem but Bayes’ and Laplace is His Prophet | Alea Deum — Mon, 18 Feb 2013 12:36:07 +0000

[...] Closet Bayesian (johndcook.com) Share this:TwitterGoogle +1FacebookLike this:LikeOne blogger likes this. This entry was posted in Humor, Philosophy of Science and tagged Bayes, Bayes Theorem, Bayesian, Bayesian probability, Bayesianism, Biology, CNN, confidence interval, Cult, Fanaticism, Frequency probability, Frequestist, Humor, Inverse Probability, p-value, Philosophy of Science, Psycology, Religion, Ronald Fisher, significance test, Statistics by Francisco Urbano García. Bookmark the permalink. [...]

By: John

John — Thu, 31 Jan 2013 19:06:21 +0000

Rob: I have read Jaynes book and enjoyed it. It dampened my desire to look at fuzzy logic etc. since it shows that Bayesian inference is the unique solution to a set of reasonable axioms.

By: Rob

Rob — Thu, 31 Jan 2013 18:14:24 +0000

If you haven’t seen the E.T. Jaynes book, it makes an awesome read. Basically argues that Bayes is the extension of logic into the realm in which there are more than one unknown.

By: Chris

Chris — Tue, 15 Jan 2013 17:23:24 +0000

Interesting. Completely new to me. I finally need to get a bit more into statistics.

Funny thing is that I would (contrary to other posters here) say that I intuitively find the Bayesian approach not that well founded philosophically (stemming from what little I know about epistemology). But I am not too sure about the Frequentist approach either. The Bayesian approach seems more intuitive practically, though.

By: anonymouse

anonymouse — Thu, 10 Jan 2013 01:35:52 +0000

Sometimes a Bayesian method has poor coverage…

By: Rules of Thumb vs. Statistical Models, or the Misconception that Will Not Die « Dart-Throwing Chimp

Wed, 09 Jan 2013 14:24:11 +0000

[...] here’s the thing: statistics isn’t science, it’s a set of tools for doing science. The decision to use statistics does not presume either regularity in, or [...]

By: John

John — Fri, 04 Jan 2013 16:12:20 +0000

Peter: There are people who are likelihoodists. They’re a pretty small group and tend to align with Bayesians. I don’t know much about that school of thought. I’ve only met one person who was enthusiastic for it.

Likelihood inference is intuitively appealing and easy to understand. But here are a couple criticisms.

You could think of likelihood inference as Bayesian inference with uniform (improper) priors. A Bayesian might object that this puts too much prior weight on parameter values known to be unlikely or impossible. Another criticism would be that posterior mode (which is what maximum likelihood is, if you use a uniform prior) can be less robust than posterior mean.

By: Peter

Peter — Fri, 04 Jan 2013 15:48:04 +0000

John,
I recently heard someone describe himself as a ‘maximum likelihood guy’. He said that the bayesian-frequentest thing was a false choice and that maximum likelihood methods should be given their own category. Any thoughts?

By: eran

eran — Fri, 04 Jan 2013 13:23:01 +0000

You might say you are neither Frequenist nor Bayesian, you are an Opportunist, which according to Bruce Lee is best (in a slightly different context..).
E

By: Manoel

Manoel — Fri, 04 Jan 2013 12:08:05 +0000

The Nate’s book is great, but the Bayesian discussion on the book is quite bizarre. Some of the claims there doesn’t make sense. And I say that as a “Bayesian” myself (actually I agree with John on being a Bayesian). In any case, Nate Silver made claims like: if Fisher didn’t opposed Bayesian statistics, he’d accept that smoking causes cancer. Not only he doesn’t present any evidence in favor of this, but it doesn’t make sense. There is nothing special in Bayesian methods to allow you to uncover causality.

By: John Dyer

John Dyer — Thu, 03 Jan 2013 20:58:53 +0000

I supposed you like tabs in your code instead of spaces!

By: Scott

Scott — Thu, 03 Jan 2013 20:57:09 +0000

I’m almost finished reading Nate Silver’s book, ” The Signal and the Noise: Why Most Predictions Fail – But Some Don’t” and he discusses Bayesian methods quite a bit, from the perspective of making predictions. As a lapsed mathematician (I’ve been a tech writer for the last 20+ years), I found it quite accessible and it made a lot of sense to me.

By: John

John — Thu, 03 Jan 2013 20:09:36 +0000

Eric: I understand. There can be tremendous political incentives to stick with traditional methods. Scientists and bureaucrats who don’t understand statistics are extremely conservative about statistical methods. They think what ever they learned is the “right” thing to do and have no to way to evaluate anything new.

I’ve also seen pressure the other way, to use Bayesian methods just because they’re sexy. Some people go to tremendous effort — or ask other people to go to tremendous effort on their behalf– to tune a Bayesian method to behave like a comparable frequentist method. Why not just use the frequentist method they’re trying to ape? Because they get brownie points for using Bayesian methods.

By: Eric

Eric — Thu, 03 Jan 2013 19:44:40 +0000

John (re: Tomas) or just how freaking hard it can be to convince reviewers that your newfangled methods are right. I always worry if I’m going to be spending “reviewer political capital” arguing for methods that I like and they may hate/not-understand.

By: Dan

Dan — Thu, 03 Jan 2013 19:42:09 +0000

I like the tool metaphor. I’ve been asked “are you an OO programmer”. I usually respond “I’m a programmer and OO is in my toolbox”. I’m a programming paradigm omnivore. Perhaps you are statistically promiscuous…er, you know what I mean

For more on Bayes see:

The Theory That Would Not Die: How Bayes’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy – Sharon Bertsch McGrayne

An Introduction to Probability and Inductive Logic – Ian Hacking

By: John

John — Thu, 03 Jan 2013 19:17:58 +0000

Tomas: Maybe you should call yourself a pragmatic Bayesian. I agree that Bayesian statistics has a better philosophical foundation. But you then have to ask whether a Bayesian approach in practice is better than a frequentist approach in practice for your particular problem. The philosophical argument for Bayesian analysis doesn't take into account approximately specified priors, availability of quality software, how well a model is understood, time required to do an analysis, sociological and legal constraints, etc.

By: Tomas

Tomas — Thu, 03 Jan 2013 18:26:57 +0000

I am a Bayesian hypocrite. The Bayesian approach is the philosophically sounder and more precise approach. You should always use it, as long as conjugate priors work.

By: Giles Warrack

Giles Warrack — Thu, 03 Jan 2013 17:37:40 +0000

Perhaps you are a closet manual-transmissionist. I am now a closet automatic-transmissionist.

Happy New Year, GW

By: John

John — Thu, 03 Jan 2013 17:24:04 +0000

Craig: All statistical inference involves subjective assumptions; knowledge-free inference is impossible. But frequentist statistics does do a better job of hiding its subjective assumptions. There are many approaches to overcoming the objection to prior distributions. Here are a few off the top of my head.

Do a sensitivity analysis and show that conclusions are not sensitive to priors (if that's true).
Special case of sensitivity analysis: show that skeptical and optimistic priors lead to the same conclusion.
Use "objective" priors, i.e. priors chosen to satisfy some mathematical optimality condition rather than prior belief.
Use empirical Bayes.
Treat the Bayesian procedure as a black box and study its frequentist operating characteristics.

By: Craig Smuda

Craig Smuda — Thu, 03 Jan 2013 16:26:27 +0000

From the non-statistician viewpoint, help me understand ‘knowledge-free’ prior estimation and I’ll happily roll- well, provided it doesn’t involve explaining it to reviewers every time. The elegance of Bayes’ is undermined by the (at least to a non-specialist) secret-sauce nature of these techniques- often it feels like all the weird assumptions are front-loaded into the prior process, rather than secreted in the test as in frequentist methods, where the Ho/Ha setup elegantly captures the general opinion that differences are probably interesting.

By: John

John — Thu, 03 Jan 2013 15:55:13 +0000

Eric: Sometimes a frequentist estimator is very simple, requiring pencil and paper rather than MCMC simulation. Such an estimator is easier to understand, or rather gives a greater feeling of understanding. Now as to what the estimator means, that’s another matter.

By: Eric

Eric — Thu, 03 Jan 2013 15:44:36 +0000

John, can you give a few examples where the non-Bayesian approach is easier to understand? I’d love to highlight them when I’m teaching Bayesian stats.