Science

On December 9, 1969 the Washington Post ran a full-page ad that began

Mr. Nixon: You can cure cancer.

If America could put a man on the moon, she should be able to cure cancer. And why not? Well, because cancer research isn’t rocket science. (Actually, rocket science isn’t science; it’s engineering.) The science necessary to put a man on the moon was well known; the science necessary to cure cancer was not.

President Nixon was eager to comply with the request for massive funding for cancer research. However, many scientists were opposed to the idea. Cancer researcher Sol Spiegelman, for example, believed such a push was premature.

An all-out effort at this time would be like trying to land a man on the moon without knowing Newton’s laws of gravity.

James Watson warned

… we must reject the notion that we will be lucky. … Instead we will be witnessing a massive expansion of well-intentioned mediocrity.

How many scientists today would argue against a funding increase for their area of study?

Quotes taken from Emperor of all Maladies

Related post:

Not exactly rocket science

Rice University donated the land for NASA’s Johnson Space Center. However, there were strings attached. According to Rocket Men,

If NASA gives up manned space flight, however, under the terms of its lease , it will have to relinquish Houston’s Johnson Spacecraft [sic] Center back to Rice University.

I imagine NASA will always at least talk about putting people in space so they can hold on to their land.

Update: Here’s a newspaper clipping about the deal. I don’t know where it’s from or whether it’s accurate.

Related posts:

Apollo 11 wasn’t perfect
Not exactly rocket science
After two days, I’d turned into an idiot

Ever wonder why astronauts schedules are crammed with activity? A simple explanation is that time in space is a very limited commodity and so they naturally want to accomplish as much as possible. While that’s undoubtedly true, there’s also another reason.

Early in the space program, a NASA psychiatrist spent two days in an isolation tank with scuba gear to experience simulated weightlessness.

I thought a little, and then I stopped thinking altogether. … incredible how idleness of body leads to idleness of mind. After two days, I’d turned into an idiot. That’s the reason why, during a flight, astronauts are always kept busy.

From Rocket Men.

Related post:

Not exactly rocket science

From Magic School Bus:

Take chances, make mistakes, and get messy.

Magic School Bus is an educational television show for children. The quote above is often repeated by the main character of the show, Ms. Frizzle.

Too many programs that supposedly teach science only teach results from science. Magic School Bus does both. It teaches specific facts, such as the names of the planets, but it also teaches that science is about taking chances, making mistakes, and getting messy.

Related post:

Preparing for innovation

Headlines are saying today that NASA found microbes that use arsenic the way all other known life uses phosphorous. The NASA web site says NASA-Funded Research Discovers Life Built With Toxic Chemical. Some other headlines include “NASA finds ‘alien life’ made of arsenic,” “NASA finds arsenic-based life,” and “NASA finds arsenic-loving bacterium.” These headlines are misleading.

The phrase arsenic-based life is misleading because most people would assume this is in contrast to carbon-based life. No, the discovery involves substituting arsenic for phosphorous. So this new microbe is only arsenic-based in the sense that most life is phosphorous-based. Actually, even that is not correct. This is a phosphorous-based life form that has been tricked into using arsenic.

NASA did not find a microbe that substitutes arsenic for phosphorous. They coaxed a microbe into substituting arsenic for phosphorous. Here’s the relevant paragraph from NASA’s story:

The newly discovered microbe, strain GFAJ-1, is a member of a common group of bacteria, the Gammaproteobacteria. In the laboratory, the researchers successfully grew microbes from the lake on a diet that was very lean on phosphorus, but included generous helpings of arsenic. When researchers removed the phosphorus and replaced it with arsenic the microbes continued to grow. Subsequent analyses indicated that the arsenic was being used to produce the building blocks of new GFAJ-1 cells.

So it seems that NASA found a microbe that could use arsenic, not a microbe that naturally does use arsenic. Perhaps some are inferring that because NASA was able to make this happen in a lab, it may also have happened naturally, though no one has seen that. Maybe so.

NASA goes on to say

The key issue the researchers investigated was when the microbe was grown on arsenic did the arsenic actually became incorporated into the organisms’ vital biochemical machinery, such as DNA, proteins and the cell membranes.

This is an amazing discovery, but it’s not quite the discovery that headlines imply.

Update: More detailed criticism of the NASA announcement from Nature News. Experts challenge the claim that the microbes actually incorporate arsenic in organic compounds.

From Kevin Kelly’s book What Technology Wants:

Our body size is, weirdly, almost exactly in the middle of the size of the universe. The smallest things we know about are approximately 30 orders of magnitude smaller than we are, and the largest structures in the universe are about 30 orders of magnitude bigger.

Related posts:

There isn’t a googol of anything
Means and inequalities
Logarithms, music, and arsenic

David Freedman gives two examples of animal-to-human studies that went horribly wrong. One actually happened. The other is hypothetical.

The actual study involves the experimental drug TGN1412. The compound was found safe in animal studies at 500 times the dose that would be given to humans. In 2006, TGN1412 was administered to six healthy men. All six were in excruciating pain within an hour of receiving the drug. Within 48 hours, all six were experiencing multiple organ failure. One subject remained in intensive care for several months. More information is available in this report.

Safety in animal studies is necessary but insufficient for testing new compounds in human subjects.That is, compounds that are harmful to animals do not go on to testing in human subjects. This policy is eminently reasonable. However, some drugs that would have been safe and effective in humans are discarded because they were toxic in animals. From Freedman:

It is frequently claimed that penicillin might easily have become one of those mistakenly discarded drugs because it sickens rabbits and guinea pigs in large or in oral doses.

In other words, animal testing might have blocked the development of one of the most important drugs in the history of medicine.

How well can you predict height based on genetic markers?

A 2009 study came up with a technique for predicting the height of a person based on looking at the 54 genes found to be correlated with height in 5,748 people — and discovered the results were one-tenth as accurate as the 125–year-old technique of averaging the heights of both parents and adjusting for sex.

The quote above is from Wrong: Why experts keep failing us — and how to know when not to trust them by David Freedman.

The article Freedman quotes is Predicting human height by Victorian and genomic methods. The “Victorian” method is the method suggested by Sir Francis Galton of averaging parents’ heights. The article’s abstract opines

For highly heritable traits such as height, we conclude that in applications in which parental phenotypic information is available (eg, medicine), the Victorian Galton’s method will long stay unsurpassed, in terms of both discriminative accuracy and costs.

Related posts:

Why heights are normally distributed
Why heights are not normally distributed

Years ago, Dentyne chewing gum ran an advertising campaign with the line “four out of five dentists surveyed recommend sugarless gum for their patients who chew gum.” Of course there’s no mention of sample size. Maybe “four out of five” meant 80% of a large survey, or maybe they literally surveyed five dentists.

Even if they only talked to five dentists, you’d think that if four dentists out of five came to the same conclusion, it is quite likely that they have good advice. Individuals have their biases, but if a large majority comes to the same conclusion independently, maybe some underlying truth is responsible for the consensus rather than a coincidence of prejudices.

However, there is a fallacy in the preceding argument. It implicitly assumes that professionals make up their minds independently and that their prejudices are independent. That may be true on some small objective problem. Several scientists may conduct independent experiments and have independent errors. In that case, if most agree on a measurement, that measurement is likely to be accurate. But ask a group of scientists working in the same area if their area deserves more funding. Of course they’ll agree. Their financial interests are highly correlated.

James Surowiecki’s book The Wisdom of Crowds argues that crowds can be amazingly intelligent. Crowds can also be incredibly foolish. One of the necessary conditions for crowd wisdom is independence. The book gives examples of experiments in which the average independent estimates, such as the weight of a cow or the number of jelly beans in a jar, surprisingly accurate. But if there were an open debate rather than an anonymous poll, the estimates would no longer be independent.  If one influential persons offers a guess, other estimates will be anchored by that guess and tend to confirm it.

William Briggs has an excellent article this morning on scientific consensus. The context of his article is climate change, though I don’t want to open a debate here on climate change. For that matter, I don’t want to open a debate on the merits of sugarless chewing gum. I’m more interested in what the article says about how a consensus becomes self-reinforcing.

In his autobiography, The Pleasures of Statistics, Frederick Mosteller gives an amusing example of why observational studies are no substitute for doing experiments.

We are all familiar with the idea that we can estimate height in male adults from their weight. … But not one of us believes that adding 20 pounds by eating and minimizing exercise will add an inch to our height.

The problem is not simply that the direction of causality backward, it’s that we cannot use a static description to predict what will happen if we change something.

Although regression situations may give one the illusion of finding out what would happen if we changed something, in the absence of an experiment they offer merely offer guesses.

He summarizes his point by quoting George Box:

To find out what happens to a system when you interfere with it, you have to interfere with it (and not just passively observe it).

Remember this next time you hear claims such as every dollar spent on X saves so many dollars spent on Y. Or every minute spent exercising increases your life expectancy by so many minutes. Or every time you do some activity you increase or decrease your risk of cancer by so much. First of all, these kinds of statements are linear extrapolations on situations that are not linear. Second, they may be observations that do not describe what will happen when you change something. They may be no more true than the idea that gaining weight makes you taller.

Here’s an example of how observation and intervention differ. Lottery winners often go bankrupt within a couple years of receiving their prize. If you suddenly make someone a millionaire, they’re not a typical millionaire.

Related posts:

Numerator-only data
Randomized trials of parachute use

When I was a little kid, I asked some adults the following question.

If hot things cool, and cool things warm up, could something hot cool down and warm back up?

The people I asked didn’t understand my question and just laughed. I have no idea how old I was, but I wasn’t old enough to articulate what I was thinking.

Here’s what I had in mind. I knew that hot things like a cup of coffee grew cold. And I knew that cold things, say a glass of milk, get warm. Well, could the coffee get so cold that it becomes a cold thing and start to warm back up?

Could the coffee become as cold as the glass of milk? Common sense suggests that can’t happen. When we say coffee grows cold, we mean that it becomes relatively colder, closer to room temperature. And when we say the milk is getting warm, we also mean it is getting closer to room temperature. We’ve never left a hot cup of coffee on a table and come back later to find that it has cooled off so much that it is colder than room temperature. But could there be small fluctuations?

As the coffee and milk head toward room temperature, could they overshoot the target, just by a little bit? Say room temperature is 70 °F, the coffee starts out at 150 °F, and the milk starts out at 40 °F. We don’t expect the coffee to cool down to 40 °F or the milk to warm up to 150 °F. But could the coffee cool down to 69.5 °F and then go back up to 70 °F? Could the milk warm up to 70.5 °F and then cool back down to 70 °F?

I didn’t get a satisfactory answer to my childhood question until I was in college. Then I found out about Newton’s law of cooling. It says that the rate at which a warm body cools is proportional to the difference between its current temperature and the ambient temperature. This law can be written as a differential equation whose solution shows that the temperature of a warm body decreases exponentially to the ambient temperature. The temperature curve always slopes downward. It doesn’t wiggle even a little on its journey to room temperature. Cold bodies warm up the opposite way, exponentially approaching room temperature but never exceeding it.

In case this seems obvious, think about thermostats. They don’t work this way. Say the temperature in a room is 85 °F and you’d like it to be 72 °F, so you turn on the air conditioning. Will the temperature steadily lower to 72 °F? Not exactly. If you were to plot the temperature in the room over time and look at the graph from far enough away, it would look like it is steadily going down to the desired temperature. But if you look at the graph more closely, you’ll see wiggles. The AC may cool the room to a little below 72 °F, maybe to 70 °F. The AC would cut off and the temperature would rise to 72 °F. Unlike the cup of hot coffee, the AC will often overshoot its target, though not by too much. The temperature may feel constant, but it is not. It oscillates around the desired temperature.

The video below features a demonstration that lightning is as likely to strike wood as metal.

I want to focus on one line from the video. After showing simulated lightning strikes that hit a wooden rod five times and a copper rod five times, the narrator says

It’s five all, proof that metal does not attract lightning.

No, such an experiment would prove no such thing. I imagine the researchers conducted a much larger experiment and selected a representative sample. And I’m willing to accept their conclusion that metal does not attract lightning. But I would not accept such a conclusion from an experiment with 10 samples. What the experiment proves is that, under their experimental conditions, lightning will sometimes strike wood even while a metal rod is nearby.

I have two complementary criticisms of this made-for-video science.

1. The results could easily happen if their conclusion were not true.
2. The results could easily not have happened if there conclusion were true.

Suppose in reality, lightning will not always strike the metal rod, but will prefer the metal. Suppose in the long run, lightning will strike the metal rod 60% of the time. It would not be unusual in that case to do an experiment with 10 strikes and find that half or more of the strikes hit wood.

Now suppose the researchers are exactly correct. In the long run, lightning has no preference for one rod or the other. What would viewers have thought if they showed a clip of 10 strikes, of which 6 hit metal and 4 hit wood? Many would have howled in protest. If lightning really had no preference for metal, the result should have been an even split, right? This is an example of the Law of Small Numbers. People underestimate the variability of small samples.

If the probability of lightning striking each rod is 50%, then in a sequence of experiments each containing 10 strikes, most will not have an exact 5-5 split. If you flip 10 fair coins, the most likely outcome is a 5-5 split, but this will happen only about 1/4 of the time. It’s more likely that you’ll get near a 5-5 split, sometimes with more heads and sometimes with more tails.

The exact 5-5 split in the video is good showmanship, but it’s misleading science.

Related posts:

Law of small numbers
Example of the law of small numbers
Law of medium numbers

There’s a law of large numbers, a law of small numbers, and a law of medium numbers in between.

The law of large numbers is a mathematical theorem. It describes what happens as you average more and more random variables.

The law of small numbers is a semi-serious statement about about how people underestimate the variability of the average of a small number of random variables.

The law of medium numbers is a term coined by Gerald Weinberg in his book An Introduction to General Systems Thinking. He states the law as follows.

For medium number systems, we can expect that large fluctuations, irregularities, and discrepancy with any theory will occur more or less regularly.

The law of medium numbers applies to systems too large to study exactly and too small to study statistically. For example, it may be easier to understand the behavior of an individual or a nation than the dynamics of a small community. Atoms are simple, and so are stars, but medium-sized things like birds are complicated. Medium-sized systems are where you see chaos.

Weinberg warns that medium-sized systems challenge science because scientific disciplines define their boundaries by the set of problems they can handle. He says, for example, that

Mechanics, then, is the study of those systems for which the approximations of mechanics work successfully.

He warns that we should not be mislead by a discipline’s “success with systems of its own choosing.”

Weinberg’s book was written in 1975. Since that time there has been much more interest in the emergent properties of medium-sized systems that are not explained by more basic sciences. We may not understand these systems well, but we may appreciate the limits of our understanding better than we did a few decades ago.

Related posts:

Laws of large numbers and small numbers
Gerald Weinberg’s law of twins
Subnatural and supernatural

The more active a research area is, the less reliable its results are.

John Ioannidis suggested popular areas of research publish a greater proportion of false results in his paper Why most published research findings are false. Of course popular areas produce more results, and so they will naturally produce more false results. But Ioannidis is saying that they also produce a greater proportion of false results.

Now Thomas Pfeiffer and Robert Hoffmann have produced empirical support for Ioannidis’s theory in the paper Large-Scale Assessment of the Effect of Popularity on the Reliability of Research. Pfeiffer and Hoffmann review two reasons why popular areas have more false results.

First, in highly competitive fields there might be stronger incentives to ‘‘manufacture’’ positive results by, for example, modifying data or statistical tests until formal statistical significance is obtained. This leads to inflated error rates for individual findings: actual error probabilities are larger than those given in the publications. … The second effect results from multiple independent testing of the same hypotheses by competing research groups. The more often a hypothesis is tested, the more likely a positive result is obtained and published even if the hypothesis is false.

In other words,

1. In a popular area there’s more temptation to fiddle with the data or analysis until you get what you expect.
2. The more people who test an idea, the more likely someone is going to find data in support of it by chance.

The authors produce evidence of the two effects above in the context of papers written about protein interactions in yeast. They conclude that “The second effect is about 10 times larger than the first one.”

Related posts:

Why microarray conclusions are so often wrong
Using Photoshop on experimental results
Irreproducible analysis
Make up your own rules of probability