The Economist posted an article online this weekend about the scandal over irreproducible cancer research by Anil Potti. My colleagues Keith Baggerly and Kevin Coombes have been crying foul about this since 2007. I first blogged about it in January 2008.
The story started getting widespread attention last summer when the Cancer Letter reported that Dr. Potti had lied on grant applications. Since then there have been articles in the popular press, and people are staring to file lawsuits.
Apparently the tipping point in the story was finding a fib on Potti’s resume. According to The Economist
He falsely claimed to have been a Rhodes Scholar in Australia (a curious claim in any case, since Rhodes scholars only attend Oxford University).
So what finally got people to pay attention was not accusations of incompetent or fraudulent science, but résumé padding. As Keith Baggerly commented,
I find it ironic that we have been yelling for three years about the science, which has the potential to be very damaging to patients, but that was not what has started things rolling.
12 thoughts on “Bad science is tolerable, résumé padding is not”
I think that the rule is as follows. You can cheat, but you should do it with plausible deniability.
Al Capone could have said “organized crime is tolerable, fiscal fraud is not”. In both cases, what matters is what is easier to prove…
A two-part reponse:
I disagree somewhat with the analogy to the case of Al Capone. As for Al Capone, I’m willing to believe the tax fraud prosecution was done because it was easier to prove. But the goal there was to imprison someone whom the authorities suspected of numerous bad actions. The bad actions were well known, but difficult to attribute to Al Capone.
In the Anil Potti case, only a few were aware of the bad science; none of them were close to analagous to the FBI or anyone in any authority. The few who were aware of the bad science brought it to the attention of Anil Potti and his colleague (and mentor, if memory serves) Joseph Nevins. When they did not correct it, it was brought to the attention of Duke officials as well as the editors of jouranls in which the bad science was published. None of these authorities believed any bad science was even being done, let alone concerned themselves with who was to blame or how to correct the situation. It was only when Anil Potti’s claim to having been a Rhodes scholar was shown to be false did anyone in authority even consider if bad science were being done. I probably should note that there is some evidence that some folks at the NCI or NIH were aware of or stongly suspected the bad science, but their concerns were kept confidential until after the cat was out of the bag.
To make it analagous to Al Capone’s case, the situation would have involved a couple of legitimate businessmen (no irony intended) discovering that it appeared that Al Capone was involved in serious crime (murder, extortion, bootlegging and so on) perhaps without being aware of it. These businessmen would have warned Al Capone of the apparent problem, with him responding by denying any crimiality. These businessmen would have begun alerting various law enforcement officials, as well as business partners of Al Capone, with none of them being willing to suspect there was any criminality. Perhaps one Federal agency had begun to suspect his crimeas and to investigate, but kept this information confidential. Finally, an investigative journalist discovered that Al Capone was a tax cheat, and only then did anyone in authority pay attention to Al Capone’s alleged crimes.
At that point someone might say that it was ironic that a couple of businessmen had been yelling for three years about Al Capone’s murders, extortion, bootlegging, and so on, but what finally got the investigation going was his tax affairs.
That is very different from having a host of agencies being aware that Al Capone was a serious criminal but being unable to prove a case against him for anything but tax fraud.
The second part:
Although there is plenty of irony to be found in the Anil Potti investigation, I for one was dismayed but not suprised that none of it was taken seriously by the majority of the academy until he was alleged to have made a false claim to a credential. The academy lives and dies by credentials. What is the number one necessary condition for publication? Credentials. What is the number one necessary qualification for employment? Credentials. What is the most basic atom of information about any academic? Credentials. Remember, ‘publish or perish’ applies only to those who already have the requisite credentials. I would bet that one could be the next Leonardo or Einstein and produce amazing, ground-breaking and paradigm-changing work, but without a degree from an accredited institution, one would have no hope of publishing, let alone a position in the academy. On the other hand, with a Ph.D. from a top-ten graduate program, one could easily publish mediocre, obscure, useless, or outright fraudulent results without too much trouble, as has been proven time and time again.
Just look at the Mathematics Genealogy Project. One might think on first hearing about it that it is a record of the development of ideas, fields of study, or something to do with mathematics. It is only formally concerned with mathematics — it is only concerned with the conferral of Ph.D. degrees by holders of Ph.D. degrees. Incidentally these degrees are in mathematics, or a related field. It has absolutely nothing to do with the work performed to induce the induction into the fraternity, nor with what may or may not have been learned from the degree conferrer, nor anything to do with mathematics itself, beyond the possible inclusion of the dissertation title in question.
It is telling to me that the placement of individuals in the context of mathematical research concerns credentials, and credentials only, and not one whit about the value or originality of mathematical research, or even whether it is true ot not!
I’ve heard Keith Baggerly give an hour long talk on this, and am still shocked that this has been allowed to persist. Baggerly’s talk was clear and stunning, so I don’t think that anyone can accuse him of being a poor messenger.
The editors at Nature Medicine, the administrators at Duke, and the FDA administrators need to be called to the carpet on this. Not as faceless bureaucrats, but with individual names. There’s more than enough shame to go around here, and if the leaders of these institutions had carried out their jobs, this would have been stopped much sooner. Cowardice is no excuse for their behavior.
The Nature journals (and Science as well) are perpetually publishing and hyping low-quality results like this. Not only do they have a short-term self-interest in silencing the criticisms of shoddy but showy science, but they appear to be hewing closer to their short-term self-interest than the interests of the research community.
I guess people just find it easier to disprove the claim that someone was a Rhodes Scholar in Australia, than to identify “bad science”. The latter is a fuzzy concept and requires expertise.
Obviously, the analogy with Al Capone was only intended to be vague. Even as a scientist considering dishonest science as a treason, I’d prefer to work with a dishonest scientist than with Al Capone :-) . However, in both cases, the jury deciding is composed of non-specialist, and it is difficult to sort out prejudice from the “accusation”, wrongdoing, and normal errors, which is crucial. I guess the résumé padding has been found by someone motivated by the scientific misconduct problem.
It is sometimes really difficult to distinguish scientific dispute for scientific dishonesty.
@John Venier In my experience, the assumptions of your second part are false, particularly with respect to mathematics. Credentials are neither necessary nor sufficient for publication. I have a BS from a second tier state school, and a number of publications (none earth shattering), and I know people who have impeccable credentials and have a hard time being published (they have a hard time writing clearly). Someone who did important work that was recognized without credentials is Emil Ehrhart, who laid the foundations of enumerating lattice points in polytopes that underlies John Cook’s post on counting magic squares while a high school (lycee) teacher (he later got a PHD after retiring from teaching.)
Credentials are important for employment, but this is because the administration of the university, and the various accreditation bureaucrats are incapable of judging your competence otherwise. Their preference for top ten degrees is the academic equivalent of “Noone was ever fired for buying IBM”. This is also the reason that nothing was done until resume padding was discovered, the people who were capable of decisive action were not competent to fully weigh the evidence of fraudulent science quickly, but resume padding can be verified with a couple of phone calls.
So John, I read the Economist article but did not read the Nature stuff originally. Why would the Potti group start to do clinical trials if they knew the science was fabricated?
Also, what is Dr Potti saying about himself nowadays? That he was a victim of [X]? Or is he saying, “I straight-up lied about A, B, and C.”
What is Dr. Potti saying about himself? That’s interesting. He hired a PR firm to flood the web with positive statements to try to counteract the bad press over the scandals. They planted numerous web sites containing variations on this name in the URL and used various other SEO tricks. And it worked. These sites fill most of the top search results. They say things like he is “a compassionate Physician in the United States and cares for patients suffering with cancer.” The sites list his honors (sans the Australian Rhodes scholarship) and talk about his charity work etc.
One of Potti’s studies, in PNAS, has now been retracted.
— Everybody embellishes their curruculum vitae a little.
— Yes, but are you Batman or not?