Science

The Economist posted an article online this weekend about the scandal over irreproducible cancer research by Anil Potti. My colleagues Keith Baggerly and Kevin Coombes have been crying foul about this since 2007. I first blogged about it in January 2008.

The story started getting wide-spread attention last summer when the Cancer Letter reported that Dr. Potti had lied on grant applications. Since then there have been articles in the popular press, and people are staring to file lawsuits.

Apparently the tipping point in the story was finding a fib on Potti’s resume. According to The Economist

He falsely claimed to have been a Rhodes Scholar in Australia (a curious claim in any case, since Rhodes scholars only attend Oxford University).

So what finally got people to pay attention was not accusations of incompetent or fraudulent science, but résumé padding. As Keith Baggerly commented,

I find it ironic that we have been yelling for three years about the science, which has the potential to be very damaging to patients, but that was not what has started things rolling.

Related posts:

Popular research areas produce more false results
Using Photoshop on research results
Highlights from Reproducible Ideas

There’s a major divide between the way scientists and programmers view the software they write.

Scientists see their software as a kind of exoskeleton, an extension of themselves. Think Dr. Octopus. The software may do heavy lifting, but the scientists remain actively involved in its use. The software is a tool, not a self-contained product.

Programmers see their software as something they will hand over to someone else, more like building a robot than an exoskeleton. Programmers believe it’s their job to encapsulate intelligence in software. If users have to depend on programmers after the software is written, the programmers didn’t finish their job.

I work with scientists and programmers, often bridging the gaps between the two cultures. One point of tension is defining when a project is done. To a scientist, the software is done when they get what they want out of it, such as a table of numbers for a paper. Professional programmers give more thought to reproducibility, maintainability, and correctness. Scientists think programmers are anal retentive. Programmers think scientists are cowboys.

Programmers need to understand that sometimes a program really only needs to run once, on one set of input, with expert supervision. Scientists need to understand that prototype code may need a complete rewrite before it can be used in production.

The real tension comes when a piece of research software is suddenly expected to be ready for production. The scientist will say “the code has already been written” and can’t imagine it would take much work, if any, to prepare the software for its new responsibilities. They don’t understand how hard it is for an engineer to turn an exoskeleton into a self-sufficient robot.

Related posts:

Good, fast, cheap: Can you really pick two?
Buggy code is biased code

I recently received review copies of the Manga Guides to physics and the universe. These made a better impression than the relativity guide that I reviewed earlier. The guide to physics has been out for a while. The guide to the universe comes out June 24.

The Manga Guide to Physics basically covers force, momentum, and energy. The pace is leisurely. There’s not much back story; it cuts to the chase fairly quickly.This guide will not prepare you to solve physics problems, but it does give you a good overview of the basics.

(These books are not entirely manga; all three books I’ve seen in the series have several pages of more traditional textbook content.)

The Manga Guide to the Universe gives a tour of cosmology from the geocentric view to current theories. It contains some very recent material, such as references to the WMAP project.

This book is more rushed than the physics guide. That’s to be expected considering its ambitious scope. It devotes a fairly large amount of space to the back story and this contributes to the book being rushed.

I mentioned in my review of The Manga Guide to Relativity that although Americans associate cartoons with children, that book was not written for children. The physics guide, however, would be appropriate for a wide range of readers. Young readers may not fully appreciate the content, but they would not find anything offensive.

The Manga Guide to the Universe is inoffensive with one exception. There are a couple provocative frames in the prologue that will keep the book off some school library shelves.

From Nobel physicist Paul Dirac:

I understand what an equation means if I have a way of figuring out the characteristics of its solution without actually solving it.

According to Richard Feynman, the most important event of the 19th century was the discovery of the laws of electricity and magnetism.

From a long view of the history of mankind — seen from, say, ten thousand years from now — there can be little doubt that the most significant event of the 19th century will be judged as Maxwell’s discovery of the laws of electrodynamics. The American Civil War will pale into provincial insignificance in comparison with this important scientific event of the same decade.

From The Feynman Lectures on Physics, Volume 2.

Related post:

Grand unified theory of 19th century math

A few days ago I got a review copy of The Manga Guide to Relativity. This is an English translation of a book first published in Japanese a couple years ago.

I assume the intended audience, at least for the original Japanese edition, is familiar with manga and wants to learn about relativity. I came from the opposite perspective, more familiar with relativity than manga, so I paid more attention to the background than the foreground. My experience was more like reading The Relativity Guide to Manga.

I expected The Manga Guide to Relativity to be something like The Cartoon Guide to Genetics. However, the former has much less scientific content than the latter. A fair amount of the relativity book is background story, and the substantial parts are repetitive. As I recall, the genetics book was much more dense with information, though presented humorously.

Some parents and teachers will buy The Manga Guide to Relativity to introduce children to science in an entertaining genre. These folks may be surprised to discover the sexual undertones in the book. Americans typically equate comics with children, but the book was originally written for a Japanese audience that does not have the same view.

Donald Knuth explains how he combines theory and practice:

This has always been the main credo of my professional life. I have always tried to develop theories that shed light on the practical things I do, and I’ve always tried to do a variety of practical things so that I have a better chance of discovering rich and interesting theories. It seems to me that my chosen field, computer science — information processing — is a field where theory and practice come together more than in any other discipline, because of the nature of computing machines. …

History teaches us that the greatest mathematicians of past centuries combined theory and practice in their own careers. …

The best theory is inspired by practice. The best practice is inspired by theory.

Taken from Selected Papers on Computer Science.

Related post:

Works in the field, not in the lab
Works well versus well understood

From The Number Mysteries:

Some of the deadliest viruses in the biological books — from influenza to herpes, from polio to the AIDS virus — are constructed using the shape of an icosahedron.

An icosahedron is a regular solid with 20 triangular faces.

In case you’re curious, here’s how I make the icosahedron image.

This weekend I’ve been wrapping up unfinished projects. One of those projects was reading Kraken: The Curious, Exciting, and Slightly Disturbing Science of Squid.

The book is exactly what you might expect from the title: a quirky little book about squid. I didn’t particularly enjoy it, but that’s my fault. I just wasn’t as interested in reading a quirky little book about squid as I thought I would when the publisher offered me a copy. Squid are bizarre creatures, and some other time I might enjoy reading more about them.

The title is terrific. I probably wouldn’t have given the book a second thought if it had been entitled, for example, Teuthology. And the title isn’t just sensational; squid really are curious, possibly exciting, and at least slightly disturbing.

Computer scientist Matt Welsh said that one reason he left Harvard for Google was that he was spending 40% of his time chasing grants. At Google, he devotes all his time to doing computer science. Here’s how he describes it in his blog post The Secret Lives of Professors:

The biggest surprise is how much time I have to spend getting funding for my research. Although it varies a lot, I guess that I spent about 40% of my time chasing after funding, either directly (writing grant proposals) or indirectly (visiting companies, giving talks, building relationships). It is a huge investment of time that does not always contribute directly to your research agenda — just something you have to do to keep the wheels turning.

According to this Scientific American editorial, 40% is typical.

Most scientists finance their laboratories (and often even their own salaries) by applying to government agencies and private foundations for grants. The process has become a major time sink. In 2007 a U.S. government study found that university faculty members spend about 40 percent of their research time navigating the bureaucratic labyrinth, and the situation is no better in Europe.

Not only do scientists on average spend a large amount of time pursuing grants, they tend to spend more time on grants as their career advances. (This has an element of tautology: you advance your career in part by obtaining grants, so the most successful are the ones who have secured the most grant money.)

By the time scientists are famous, they may no longer spend much time actually doing science. They may spend nearly all their research time chasing grants either directly or, as Matt Welsh describes, indirectly by traveling, speaking, and schmoozing.

The US Supreme Court’s criticism of significance testing has been in the news lately. Here’s a criticism of significance testing involving the US Congress. Consider the following syllogism.

1. If a person is an American, he is not a member of Congress.
2. This person is a member of Congress.
3. Therefore he is not American.

The initial premise is false, but the reasoning is correct if we assume the initial premise is true.

The premise that Americans are never members of Congress is clearly false. But it’s almost true! The probability of an American being a member of Congress is quite small, about 535/309,000,000. So what happens if we try to salvage the syllogism above by inserting “probably” in the initial premise and conclusion?

1. If a person is an American, he is probably not a member of Congress.
2. This person is a member of Congress.
3. Therefore he is probably not American.

What went wrong? The probability is backward. We want to know the probability that someone is American given he is a member of Congress, not the probability he is a member of Congress given he is American.

Science continually uses flawed reasoning analogous to the example above. We start with a “null hypothesis,” a hypothesis we seek to disprove. If our data are highly unlikely assuming this hypothesis, we reject that hypothesis.

1. If the null hypothesis is correct, then these data are highly unlikely.
2. These data have occurred.
3. Therefore, the null hypothesis is highly unlikely.

Again the probability is backward. We want to know the probability of the hypothesis given the data, not the probability of the data given the hypothesis.

We can’t reject a null hypothesis just because we’ve seen data that are rare under this hypothesis. Maybe our data are even more rare under the alternative. It is rare for an American to be in Congress, but it is even more rare for someone who is not American to be in the US Congress!

I found this illustration in The Earth is Round (p < 0.05) by Jacob Cohen (1994). Cohen in turn credits Pollard and Richardson (1987) in his references.

Related posts:

How insignificant is significance testing?
Five criticisms of significance testing
Most published research results are false
Classical statistics in a nutshell

Radiation units are confusing for three or four reasons.

1. There are different units depending on whether you’re measuring how much radiation is being emitted or measuring how much is being received.
2. There are different ways of quantifying the amount of radiation received depending on whether you’re doing physics or biology.
3. For each of these measurements there are traditional units and SI units.

If you’re not familiar with scientific units, a fourth source of confusion is the prefixes for various powers of 10: milli-, micro-, etc.

The amount of radioactivity emitted by a source is measured in Becquerels or Curies. The SI unit the becquerel (Bq), one decay per second. The traditional unit Curie (Ci) is 3.7 × 10¹⁰ Bq and is about the radioactivity of a gram of radium.

The amount of radiation received by a source is measured in grays or rads. The SI unit Gray (Gy) corresponds to one joule of energy absorbed by one kilogram of matter. The traditional unit rad is 0.01 Gy.

The biological effect of radiation is measured in Sieverts or rems. Biologically effective dose is the amount of radiation received multiplied by the relative biological effectiveness (RBE) of the type of radiation source. For x-rays, the RBE is 1. For alpha rays, the RBE is 20. The SI unit of effective dose is the Sievert (Sv), which corresponds to one Gy of x-rays. A rem is 0.01 Sv.

Another unit of effect is the banana equivalent dose. A banana is 0.0001 mSv, or roughly the effective dose of radiation from eating a banana.

The following paragraph is from the science fiction novel A Canticle for Leibowitz:

A fourth century bishop and philosopher. He [Saint Augustine] suggested that in the beginning God created all things in their germinal causes, including the physiology of man, and that the germinal causes inseminate, as it were, the the formless matter — which then gradually evolved into the more complex shapes, and eventually Man. Has this hypothesis been considered?

A Canticle for Leibowitz is set centuries after a nuclear holocaust. The war was immediately followed by the “Simplification.” Survivors rejected all advanced technology and hunted down everyone who was even literate. At this point in the book, a sort of Renaissance is taking place. The question above is addressed to a scientist who is explaining some of the (re)discoveries taking place. The scientist’s response was

“I’m afraid it has not, but I shall look it up,” he said, in a tone that indicated he would not.

Was the reference to Augustine simply made up for the novel, or is there something in Augustine’s writings that the author is alluding to? If so, does anyone know what in particular he may be referring to? Is such a proto-Darwinian reading of Augustine fair?

Here’s a little saying that irritates me:

Absence of evidence is not evidence of absence.

It’s the kind of thing a Sherlock Holmes-like character might say in a detective novel. The idea is that we can’t be sure something doesn’t exist just because we haven’t seen it yet.

What bothers me is that the statement misuses the word “evidence.” The statement would be correct if we substituted “proof” for “evidence.” We can’t conclude with absolute certainty that something doesn’t exist just because we haven’t yet proved that it does. But evidence is not the same as proof.

Why do we believe that dodo birds are extinct? Because no one has seen one in three centuries. That is, there is an absence of evidence that they exist. That is tantamount to evidence that they do not exist. It’s logically possible that a dodo bird is alive and well somewhere, but there is overwhelming evidence to suggest this is not the case.

Evidence can lead to the wrong conclusion. Why did scientists believe that the coelacanth was extinct? Because no one had seen one except in fossils. The species was believed to have gone extinct 65 million years ago. But in 1938 a fisherman caught one. Absence of evidence is not proof of absence.

Though it is not proof, absence of evidence is unusually strong evidence due to subtle statistical result. Compare the following two scenarios.

Scenario 1: You’ve sequenced the DNA of a large number prostate tumors and found that not one had a particular genetic mutation. How confident can you be that prostate tumors never have this mutation?

Scenario 2: You’ve found that 40% of prostate tumors in your sample have a particular mutation. How confident can you be that 40% of all prostate tumors have this mutation?

It turns out you can have more confidence in the first scenario than the second. If you’ve tested N subjects and not found the mutation, the length of your confidence interval around zero is proportional to N. But if you’ve tested N subjects and found the mutation in 40% of subjects, the length of your confidence interval around 0.40 is proportional to √N. So, for example, if N = 10,000 then the former interval has length on the order of 1/10,000 while the latter interval has length on the order of 1/100. This is known as the rule of three. You can find both a frequentist and a Bayesian justification of the rule here.

Absence of evidence is unusually strong evidence that something is at least rare, though it’s not proof. Sometimes you catch a coelacanth.

Related posts:

Estimating the chances of something that hasn’t happened
Complementary validation