I attended a National Cancer Institute workshop yesterday entitled “Barriers to producing well-tested, user-friendly software for cutting-edge statistical methodology.” I was pleased that everyone there realized there is a huge difference between code created for personal use and reliable software that others would willingly use. Not all statisticians appreciate the magnitude of the difference.
I was also pleased that several people at the workshop were aware of the problem of irreproducible statistical analyses. Not everyone was aware how serious or how common the problem is, but those who were aware were adamant that something needs to be done about it, such as journals requiring authors to publish the code used to analyze their data.
It seems crazy that academics who publish these kind of papers are not required to make available their data and methods, unless there are commercial considerations involved. Surely this subtly undermines the whole idea of academic integrity? It seems unclear what role academic referees have to play – are they just a formal rubber stamp? It seems little wonder that such a huge percentage of academic output is destined for obscurity the instant it is published.
John
How do I get the information presented at this conference
Pallabi
The conference consisted of several people giving short presentations. I don’t think any of the presentations were published online.
Here is a document from the NCI regarding an RFP that came out of the meeting:
http://grants.nih.gov/grants/funding/SBIRContract/PHS2009-1.doc