Comments on: Probability that a study result is true

By: CogitoErgoCogitoSum

CogitoErgoCogitoSum — Sat, 17 Apr 2010 20:04:02 +0000

Its curious that these results arent more popularly known. It seems to me that statisticians, when they have something to gain in capitalism, dont mind ignoring these results. And that these results are de-emphasized and kept on the back burner of society. Statisticians and scientists alike dont mind maintaining the illusion and delusion for the rest of society, arrogantly believing their math and their methods are not wrong. Its very typical. Whenever rational thought emerges that forces us to question our precepts, it disappears into the ether and its author is usually scorned and mocked into obscurity. (aliens built the pyramids, and other such things). Here is a good argument saying that, with all the certainty we try to build into our methods, it is still likely to be wrong… sort of makes moot all of our efforts thus far.

By: Chuck Kelly

Chuck Kelly — Mon, 15 Dec 2008 21:54:56 +0000

I’m no high-powered mathematician, but this seems counter-intuitive to me. I wonder if it’s because of the difference between a de novo hypothesis and one that’s been subjected to experimental testing. I can see a minority of dreamed-up hypotheses being correct — I’ve had a lot of those myself — but once data are accumulated and the “disproven” hypotheses weeded out, that should improve the odds of the remaining hypotheses being correct. Otherwise, what’s the point of scientific investigation? A lot of the old synapses (emphasis on OLD) are degraded by time, but I must be missing something here.

By: John

John — Mon, 01 Dec 2008 22:04:24 +0000

Manoel: Let P be the number of positive hypotheses being investigated and let N be the number of negative hypotheses. R is defined as P/N. Then the probability of picking a positive hypothesis to investigate is P/(N+P) = NR/(NR + N) = R/(R+1).

I’ve wondered why Ioannidis formulated his paper in terms of odds rather than probabilities. The latter might have been clearer. Calculating probabilities from odds isn’t hard, but it interrupts the flow of reading the paper.

By: Manoel Neto

Manoel Neto — Mon, 01 Dec 2008 21:19:08 +0000

Hello John,

Could you explain how did you concluded that “The probability of selecting a hypothesis that really is true would be R/(R+1)”? I was wondering if this result came from a binomial distribution, or even a hypergeometric one. However, in both cases I couldn’t figure out how to conclude this.

Thanks very much in advance.