Comments on: Most published research results are false

By: Interpreting statistics — The Endeavour

Interpreting statistics — The Endeavour — Fri, 13 Jan 2012 14:47:50 +0000

[...] Most published research results are false Classical statistics in a nutshell [...]

By: Woody

Woody — Tue, 03 Jan 2012 21:03:24 +0000

But, proving it is wrong can be wrong as well.

By: Vínculos marginales XVI « Notas marginales

Vínculos marginales XVI « Notas marginales — Wed, 13 Apr 2011 15:00:22 +0000

[...] El culto al p-value (o por qué una gran parte de los documentos de investigación están mal) [...]

By: True Science of False Religion? | The Science of Science

True Science of False Religion? | The Science of Science — Fri, 08 Apr 2011 17:50:26 +0000

[...] into thinking all of this sounds rational and sophisticated. An example by applied mathematician John D. Cook illustrates the fundamental problem. By the 95% logic, out of 1000 pre-publication studies of a [...]

By: Jim Moult

Jim Moult — Sun, 06 Mar 2011 17:05:40 +0000

It appears relevant to me to note that — though the .05 p-value indicates a 1 in 20 chance that the null hypothesis should not have been rejected, — in cases where one is testing whether treatment A performs better than treatment B, that the likelihood that treatment A was in fact better — when the result reported was that treatment A was better with p-value < .05 — includes a high proportion of cases in which A was at least as good. Put another way, I am unlikely to be making a truly bad decision to accept and act on selecting A in preference to B, even though I may be mistaken that it is truly better. Placing in context, some of us are primarily concerned with making a choice, and not primarily with the certainty that our choice is superior. Out certainty is not unimportant, it is just of less importance. I find the discussion and observation itself fascinating.

By: Is Statistical Hypothesis Testing Scientific? | The Science of Science

Is Statistical Hypothesis Testing Scientific? | The Science of Science — Fri, 18 Feb 2011 03:45:30 +0000

[...] and Lehrer provide an explanation for this failure, which is summarized in John Cook’s blog post: Suppose you have 1,000 totally ineffective drugs to test. About 1 out of every 20 trials will [...]

By: Reporting p-values | The Science of Science

Reporting p-values | The Science of Science — Mon, 14 Feb 2011 22:04:46 +0000

[...] Reporting p-values Posted on February 14, 2011 by somikraha In a previous article, I have already tackled the fundamental notions of frequentist statistics as violating the desiderata of science. Be that as it may, the stupendous extent of this approach’s contradiction with its own norms merits some attention. This is Ioannidis’ main argument, as also John Cook (see Cook’s blog post). [...]

By: John

John — Fri, 11 Feb 2011 22:22:40 +0000

You could say there’s a survival bias: only those experiments that meet some statistical requirement are published. But it’s more subtle than that. You’ve got to have some procedure for deciding which results are correct. I would argue that p-values are not the right filter, or that at a minimum p-values are incorrectly interpreted.

By: Adam Keck

Adam Keck — Fri, 11 Feb 2011 21:51:38 +0000

Filtering publishable results by desirable p-value (last paragraph) sounds like a classic case of Survivor Bias. Correct?

By: Laurent Bossavit

Laurent Bossavit — Fri, 11 Feb 2011 21:32:03 +0000

See also Cosma Shalizi’s post, “The Neutral Model of Inquiry”: http://cscs.umich.edu/~crshalizi/weblog/698.html

By: Tweets that mention Most published research results are false — The Endeavour -- Topsy.com

Fri, 11 Feb 2011 19:39:40 +0000

[...] This post was mentioned on Twitter by Magnus Lie Hetland, topkara, HN Firehose, Renan Birck Pinheiro, Fumihiro CHIBA and others. Fumihiro CHIBA said: memo: Most published research results are false — The Endeavour http://bit.ly/dSkd3h [...]

By: More theoretical power, less real power — The Endeavour

More theoretical power, less real power — The Endeavour — Mon, 24 Jan 2011 15:31:23 +0000

[...] posts: Most published research results are false Canonical examples from robust statistics ? [...]

By: ¿Porqué muchas de las publicaciones científicas son falsas? « El nido del Gavilán

Mon, 04 Jan 2010 17:20:58 +0000

[...] en el artículo del Plos como en esta entrada del blog “The Endeavour” se intenta buscar el culpable de este fenómeno. Todas las pruebas inculpan a la indolente [...]

By: Acerca de las pruebas de hipótesis « Apuntes de Estadística

Acerca de las pruebas de hipótesis « Apuntes de Estadística — Mon, 26 Jan 2009 21:59:42 +0000

[...] John Ioannidis: p-valores pequeños no implican una probabilidad pequeña de que la hipótesis nula sea incorrecta. En una revisión de estudios médicos se encontró que el 74% de los estudios con p-valores menores que 0.05 llegaban a conclusiones erróneas. [...]

By: John

John — Thu, 30 Oct 2008 22:02:49 +0000

Basing a conclusion on a very small subset of the world population may be legitimate. It all depends on whether the sample is representative. One of the surprising results from statistics is that the quality of an inference depends only on the size of the sample, not on the size of the population the sample was drawn from. (Assuming the population is so large that you can safely ignore the difference between sampling with and without replacement, which is true of the world population.)

By: Mauro

Mauro — Thu, 30 Oct 2008 21:13:05 +0000

Hi,

Many times I see researchs saying that “some thing” was tested in thousands of people where, compared to the world population, it is less than 0,001%… so how can I believe that the investigation is correct?

Best regards,
Mauro