Comments on: Popular research areas produce more false results

By: Jon

Jon — Thu, 19 May 2011 05:50:54 +0000

What about the idea that people working in less popular areas are either inherently less concerned about scoring big prey than those in more popular fields and may be on average more concerned with accuracy at least partly because they are pursuing real passions? Also, it’s not as if working in an uncrowded area means that funding is not an issue; on the contrary the science budgets in the US has been disproportionately slashed for more ‘basic’ research, which includes many less crowded fields. Im sure it would be hard to find control conditions where the same researcher(s) shifted research from or to a more popular field, or maybe a field changed popularity status around an established researcher, but even a small amount of longitudinal data may be very revealing.

By: NPHArd

NPHArd — Tue, 17 May 2011 18:23:04 +0000

Although with a slightly different leaning this sounds a little like Gelman’s post on research papers published in China. http://www.stat.columbia.edu/~gelman/blog/ . Anyway you slice it when there is pressure to produce results there can be problems

By: Buggy code is biased code — The Endeavour

Buggy code is biased code — The Endeavour — Tue, 19 Oct 2010 14:40:05 +0000

[...] Popular research areas produce more false results Errors in math papers not a big deal? Writes large correct programs ? X [...]

By: Janne

Janne — Sat, 20 Feb 2010 12:04:56 +0000

True, deliberate reproduction is unusual – what funding agency would want to pay for it after all? But people often use results from other groups to further their own work. And when results from different groups don’t add up, things that should work don’t, and people start looking for why that may be. Or people may need to redo part of an experiment simply to create the raw material or base for their own work.

In a large, active field there’s enough inadvertent duplication of effort and comparison between related results that there’s a good chance to find the faulty data. In a slow, sparsely worked field the chance is much lower.

By: John

John — Fri, 19 Feb 2010 18:32:54 +0000

Panos, I agree that could happen. I make that argument about errors in popular areas of mathematics here. Unfortunately, scientists don't often try to reproduce each other's results. It's a lot cheaper for a mathematician to go over a colleague's proof than for a scientist to reproduce a colleague's experiment.

By: Panos Ipeirotis

Panos Ipeirotis — Fri, 19 Feb 2010 18:21:41 +0000

There is also the counterargument: In popular areas, more people are testing the published results and identify more errors. In less popular areas, there are not that many “vigilantes” and many false results are never tested.