“Why only consider biased vs. unbiased rather, a binary choice, rather than degree of bias, a continuous choice? Efficiency is also important, and someone may reasonably accept a small amount of bias in exchange for a large increase in efficiency.”

I ask in return “If everything is inferred, why should the soundness of Baysian Inference be based on a binary notion of truth and falsity? Should not there be degrees of correctness for a theory of inference? Therefore, shouldn’t Bayesian inference be capable of supplying a proof for it’s own correctness? Yet Bayesian inference lacks the ability to ‘prove,’ should ones axioms be taken as anything but absolutely true. It also lacks the ability for universal generalization. Seems like Bayesian inference would have a tough time inferring that one should use Bayesian inference. Yet, is there a system that can do any better? Prove (or infer) that there isn’t and then I’m sold.

If someone wants to insist that a parameter is constant but unknown, fine. It’s a constant. But we’re uncertain about it’s value. If we quantify our uncertainty about that constant according to certain reasonable axioms, our uncertainties obey the laws of probability. Let’s go ahead and call them probabilities. And now you have Bayesian statistics.

You can search on “Jaynes” to find the posts where I’ve mentioned his book.

I arrived at your site via a Python-related search, and was pleased to find your Bayesian blog-site (onto which I just landed). My initiation was Edwin Jaynes’s book, which I haven’t yet finished. Jaynes is quite right about physics, and convinced at least me about the Bayesian approach to probability. Have you seen Jaynes’s book? (Sorry, I haven’t read your blogs yet.)

– George Hacken

… and become a cool and chic Bayesian?

I laughed out loud at this, because of a comment I heard from a speaker at a conference long ago … he was a Bayesian and it was a Bayesian conference. During his talk he expressed the opinion that most Bayesian conferences primarily involve sitting around drinking beer and congratulating each other on having had the good sense to become Bayesians.

One of my favorite professors, a dyed-in-the-wool Aristotelian Frequentist, likened the spread of Bayesian statistics to the pod people in “Invasion of the Body Snatchers” … first one person in the department becomes one, then a couple more, and finally the whole department. He was one of the funniest professors I’ve had of any field.

And one for the hardcore geeks:

At another conference, this one neither particularly Bayesian nor Frequentist, two statisticians gave a presentation on estimating the failure time distribution for the nuclear waste containment systems which were proposed to be installed in Yucca Mountain. Given that they were designed to last at least 10,000 years, this presented a problem for pure Frequentist methods. Naturally, they adopted Bayesian methods, which happened to be relatively new to them. Nothing unusual there. But one of them concluded with some remarks about how much he liked Bayesian methods. He mentioned that in one episode of Star Trek: The Next Generation, the character Data says something about a “Bayesian millenium” [I can't remember the reference exactly], which the speaker thought was wonderful, and ended with a loud toast to the coming Bayesian millenium!

The only elementary (i.e. pre-calculus) book on Bayesian statistics I know of is Statistics: A Bayesian Approach. Since it doesn’t use calculus, it has to give lots of tangible examples, drawing chips from bowls etc. It might help build intuition more than an advanced book.

And here comes John, letting me know that I’m no more than an old fashioned frequentist… Any suggestions on where to start reading to redo myself and become a cool and chic Bayesian?