Science

Experts often end up where they started as beginners.

If you’ve never seen the word valet, you might pronounce it like VAL-it. If you realize the word has a French origin, you would pronounce it val-A. But the preferred pronunciation is actually VAL-it.

Beginning musicians play by ear, to the extent that they can play at all. Then they learn to read music. Eventually, maybe years later, they realize that music really is about what you hear and not what you see.

Beginning computer science students think that computer science is all about programming. Then they learn that computer science is actually about computation in the abstract and not about something so vulgar as a computer. But eventually they come back down to earth and realize that 99.44% of computer science is ultimately motivated by the desire to get computers to do things.

In a beginning physics class, an instructor will ask students to assume a pulley has no mass and most students will simply comply. A few brighter students may snicker, knowing that pulleys really do have mass and that some day they’ll be able to handle problems with realistic pulleys. In a more advanced class, it’s the weaker students who snicker at massless pulleys. The better students understand a reference to a massless pulley to mean that in the current problem, the rotational inertia of the pulley can safely be ignored, simplifying the calculations without significantly changing the result. Similar remarks hold for frictionless planes and infinite capacitors as idealizations. Novices accept them uncritically, sophomores sneer at them, and experts understand their uses and limitations. (Two more physics examples.)

Here’s an example from math. Freshmen can look at a Dirac function δ(x) without blinking. They accept the explanation that it’s infinite at the origin, zero everywhere else, and integrates to 1. Then when they become more sophisticated, they realize this explanation is nonsense. But if they keep going, they’ll learn the theory that makes sense of things like δ(x). They’ll realize that the freshman explanation, while incomplete, is sometimes a reasonable intuitive guide to how δ(x) behaves. They’ll also know when such intuition leads you astray.

In each of these examples, the experts don’t exactly return to the beginning. They come to appreciate their initial ideas in a more nuanced way.

“When we travel, we travel not to see new places with new eyes; but that when we come home we see home with new eyes.” — G. K. Chesterton

Related posts:

Infinite is easier than big
Final velocity
Childhood question about heat

A recent article in The New Yorker gives numerous examples of scientific results fading over time. Effects that were large when first measured become smaller in subsequent studies. Firmly established facts become doubtful. It’s as if scientific laws are being gradually repealed. This phenomena is known as “the decline effect.” The full title of the article is The decline effect and the scientific method.

The article brings together many topics that have been discussed here: regression to the mean, publication bias, scientific fashion, etc. Here’s a little sample.

“… when I submitted these null results I had difficulty getting them published. The journals only wanted confirming data. It was too exciting an idea to disprove, at least back then.” … After a new paradigm is proposed, the peer-review process is tilted toward positive results. But then, after a few years, the academic incentives shift—the paradigm has become entrenched—so that the most notable results are now those that disprove the theory.

This excerpt happens to be talking about “fluctuating asymmetry,” the idea that animals prefer more symmetric mates because symmetry is a proxy for good genes. (I edited out references to fluctuating asymmetry from the quote to emphasize that the remarks could equally apply to any number of topics. ) Fluctuating asymmetry was initially confirmed by numerous studies, but then the tide shifted and more studies failed to find the effect.

When such a shift happens, it would be reassuring to believe that the initial studies were simply wrong and that the new studies are right. But both the positive and negative results confirmed the prevailing view at the time they were published. There’s no reason to believe the latter studies are necessarily more reliable.

Related posts:

Why microarray study conclusions are so often wrong
Popular research areas produce more false results
Five criticisms of significance testing

Occam’s razor says that if two models fit equally well, the simpler model is likely to be a better description of reality. Why should that be?

A paper by Jim Berger suggests a Bayesian justification of Occam’s razor: simpler hypotheses have higher posterior probabilities when they fit well.

A simple model makes sharper predictions than a more complex model. For example, consider fitting a linear model and a cubic model. The cubic model is more general and fits more data. The linear model is more restrictive and hence easier to falsify. But when the linear and cubic models both fit, Bayes’ theorem “rewards” the linear model for making a bolder prediction. See Berger’s paper for a details and examples.

From the conclusion of the paper:

Ockham’s razor, far from being merely an ad hoc principle, can under many practical situations in science be justified as a consequence of Bayesian inference. Bayesian analysis can shed new light on what the notion of “simplest” hypothesis consistent with the data actually means.

Related links:

How loud is the evidence?
Blog posts on Bayesian statistics

The Earth appears eight times brighter from the moon than a full moon appears from the Earth.

Source: Rocket Men

Related post:

Team Moon

On December 9, 1969 the Washington Post ran a full-page ad that began

Mr. Nixon: You can cure cancer.

If America could put a man on the moon, she should be able to cure cancer. And why not? Well, because cancer research isn’t rocket science. (Actually, rocket science isn’t science; it’s engineering.) The science necessary to put a man on the moon was well known; the science necessary to cure cancer was not.

President Nixon was eager to comply with the request for massive funding for cancer research. However, many scientists were opposed to the idea. Cancer researcher Sol Spiegelman, for example, believed such a push was premature.

An all-out effort at this time would be like trying to land a man on the moon without knowing Newton’s laws of gravity.

James Watson warned

… we must reject the notion that we will be lucky. … Instead we will be witnessing a massive expansion of well-intentioned mediocrity.

How many scientists today would argue against a funding increase for their area of study?

Quotes taken from Emperor of all Maladies

Related post:

Not exactly rocket science

Rice University donated the land for NASA’s Johnson Space Center. However, there were strings attached. According to Rocket Men,

If NASA gives up manned space flight, however, under the terms of its lease , it will have to relinquish Houston’s Johnson Spacecraft [sic] Center back to Rice University.

I imagine NASA will always at least talk about putting people in space so they can hold on to their land.

Update: Here’s a newspaper clipping about the deal. I don’t know where it’s from or whether it’s accurate.

Related posts:

Apollo 11 wasn’t perfect
Not exactly rocket science
After two days, I’d turned into an idiot

Ever wonder why astronauts schedules are crammed with activity? A simple explanation is that time in space is a very limited commodity and so they naturally want to accomplish as much as possible. While that’s undoubtedly true, there’s also another reason.

Early in the space program, a NASA psychiatrist spent two days in an isolation tank with scuba gear to experience simulated weightlessness.

I thought a little, and then I stopped thinking altogether. … incredible how idleness of body leads to idleness of mind. After two days, I’d turned into an idiot. That’s the reason why, during a flight, astronauts are always kept busy.

From Rocket Men.

Related post:

Not exactly rocket science

Headlines are saying today that NASA found microbes that use arsenic the way all other known life uses phosphorous. The NASA web site says NASA-Funded Research Discovers Life Built With Toxic Chemical. Some other headlines include “NASA finds ‘alien life’ made of arsenic,” “NASA finds arsenic-based life,” and “NASA finds arsenic-loving bacterium.” These headlines are misleading.

The phrase arsenic-based life is misleading because most people would assume this is in contrast to carbon-based life. No, the discovery involves substituting arsenic for phosphorous. So this new microbe is only arsenic-based in the sense that most life is phosphorous-based. Actually, even that is not correct. This is a phosphorous-based life form that has been tricked into using arsenic.

NASA did not find a microbe that substitutes arsenic for phosphorous. They coaxed a microbe into substituting arsenic for phosphorous. Here’s the relevant paragraph from NASA’s story:

The newly discovered microbe, strain GFAJ-1, is a member of a common group of bacteria, the Gammaproteobacteria. In the laboratory, the researchers successfully grew microbes from the lake on a diet that was very lean on phosphorus, but included generous helpings of arsenic. When researchers removed the phosphorus and replaced it with arsenic the microbes continued to grow. Subsequent analyses indicated that the arsenic was being used to produce the building blocks of new GFAJ-1 cells.

So it seems that NASA found a microbe that could use arsenic, not a microbe that naturally does use arsenic. Perhaps some are inferring that because NASA was able to make this happen in a lab, it may also have happened naturally, though no one has seen that. Maybe so.

NASA goes on to say

The key issue the researchers investigated was when the microbe was grown on arsenic did the arsenic actually became incorporated into the organisms’ vital biochemical machinery, such as DNA, proteins and the cell membranes.

This is an amazing discovery, but it’s not quite the discovery that headlines imply.

Update: More detailed criticism of the NASA announcement from Nature News. Experts challenge the claim that the microbes actually incorporate arsenic in organic compounds.

The “paleolithic diet” has gotten more press lately. Paleolithic diet advocates say our ancestors lived in a state of gastronomic innocence, eating mostly meat, before they were seduced by agriculture and fell into eating grain. Some go further and say that not only should we eat like cavemen, we should live like cavemen as well. For example, we should have random bursts of exercise, as if fleeing a saber-toothed tiger, followed by long periods of leisure.

I am amused by how much some people believe they know about paleolithic life. Most of us don’t know that much about how our great grandparents lived, and yet others make confident detailed claims about the lifestyles of pre-historic ancestors. This is convenient since their claims are unlikely to be proven wrong, given how little we know or are ever likely to know ancient lifestyles. Of course those making the boldest claims are not scientists but popularizers who take a hint from the scientists and run with it.

I have no opinion on the actual recommendations of the fans of paleolithic culture. Maybe we would be better off eating more meat or having random bursts of intense exercise; I have no idea. However, I object to the pseudo-scientific rhetoric used to support the recommendations. I also object to the implicit assumption that it would necessarily be good to emulate the lives of paleolithic humans even if we did know how they lived.

Even the little we think we know about ancient cuisine should be called into question. A paper entitled Thirty thousand-year-old evidence of plant food processing was posted online this week which suggests people were making flour 10,000 years earlier than previously thought. Of course this doesn’t mean that people all over the world were living on pasta, but it does underscore how little we know about the real paleolithic diet.

David Freedman gives two examples of animal-to-human studies that went horribly wrong. One actually happened. The other is hypothetical.

The actual study involves the experimental drug TGN1412. The compound was found safe in animal studies at 500 times the dose that would be given to humans. In 2006, TGN1412 was administered to six healthy men. All six were in excruciating pain within an hour of receiving the drug. Within 48 hours, all six were experiencing multiple organ failure. One subject remained in intensive care for several months. More information is available in this report.

Safety in animal studies is necessary but insufficient for testing new compounds in human subjects.That is, compounds that are harmful to animals do not go on to testing in human subjects. This policy is eminently reasonable. However, some drugs that would have been safe and effective in humans are discarded because they were toxic in animals. From Freedman:

It is frequently claimed that penicillin might easily have become one of those mistakenly discarded drugs because it sickens rabbits and guinea pigs in large or in oral doses.

In other words, animal testing might have blocked the development of one of the most important drugs in the history of medicine.

How well can you predict height based on genetic markers?

A 2009 study came up with a technique for predicting the height of a person based on looking at the 54 genes found to be correlated with height in 5,748 people — and discovered the results were one-tenth as accurate as the 125–year-old technique of averaging the heights of both parents and adjusting for sex.

The quote above is from Wrong: Why experts keep failing us — and how to know when not to trust them by David Freedman.

The article Freedman quotes is Predicting human height by Victorian and genomic methods. The “Victorian” method is the method suggested by Sir Francis Galton of averaging parents’ heights. The article’s abstract opines

For highly heritable traits such as height, we conclude that in applications in which parental phenotypic information is available (eg, medicine), the Victorian Galton’s method will long stay unsurpassed, in terms of both discriminative accuracy and costs.

Related posts:

Why heights are normally distributed
Why heights are not normally distributed

Years ago, Dentyne chewing gum ran an advertising campaign with the line “four out of five dentists surveyed recommend sugarless gum for their patients who chew gum.” Of course there’s no mention of sample size. Maybe “four out of five” meant 80% of a large survey, or maybe they literally surveyed five dentists.

Even if they only talked to five dentists, you’d think that if four dentists out of five came to the same conclusion, it is quite likely that they have good advice. Individuals have their biases, but if a large majority comes to the same conclusion independently, maybe some underlying truth is responsible for the consensus rather than a coincidence of prejudices.

However, there is a fallacy in the preceding argument. It implicitly assumes that professionals make up their minds independently and that their prejudices are independent. That may be true on some small objective problem. Several scientists may conduct independent experiments and have independent errors. In that case, if most agree on a measurement, that measurement is likely to be accurate. But ask a group of scientists working in the same area if their area deserves more funding. Of course they’ll agree. Their financial interests are highly correlated.

James Surowiecki’s book The Wisdom of Crowds argues that crowds can be amazingly intelligent. Crowds can also be incredibly foolish. One of the necessary conditions for crowd wisdom is independence. The book gives examples of experiments in which the average independent estimates, such as the weight of a cow or the number of jelly beans in a jar, surprisingly accurate. But if there were an open debate rather than an anonymous poll, the estimates would no longer be independent.  If one influential persons offers a guess, other estimates will be anchored by that guess and tend to confirm it.

William Briggs has an excellent article this morning on scientific consensus. The context of his article is climate change, though I don’t want to open a debate here on climate change. For that matter, I don’t want to open a debate on the merits of sugarless chewing gum. I’m more interested in what the article says about how a consensus becomes self-reinforcing.

In his autobiography, The Pleasures of Statistics, Frederick Mosteller gives an amusing example of why observational studies are no substitute for doing experiments.

We are all familiar with the idea that we can estimate height in male adults from their weight. … But not one of us believes that adding 20 pounds by eating and minimizing exercise will add an inch to our height.

The problem is not simply that the direction of causality backward, it’s that we cannot use a static description to predict what will happen if we change something.

Although regression situations may give one the illusion of finding out what would happen if we changed something, in the absence of an experiment they offer merely offer guesses.

He summarizes his point by quoting George Box:

To find out what happens to a system when you interfere with it, you have to interfere with it (and not just passively observe it).

Remember this next time you hear claims such as every dollar spent on X saves so many dollars spent on Y. Or every minute spent exercising increases your life expectancy by so many minutes. Or every time you do some activity you increase or decrease your risk of cancer by so much. First of all, these kinds of statements are linear extrapolations on situations that are not linear. Second, they may be observations that do not describe what will happen when you change something. They may be no more true than the idea that gaining weight makes you taller.

Here’s an example of how observation and intervention differ. Lottery winners often go bankrupt within a couple years of receiving their prize. If you suddenly make someone a millionaire, they’re not a typical millionaire.

Related posts:

Numerator-only data
Randomized trials of parachute use

When I was a little kid, I asked some adults the following question.

If hot things cool, and cool things warm up, could something hot cool down and warm back up?

The people I asked didn’t understand my question and just laughed. I have no idea how old I was, but I wasn’t old enough to articulate what I was thinking.

Here’s what I had in mind. I knew that hot things like a cup of coffee grew cold. And I knew that cold things, say a glass of milk, get warm. Well, could the coffee get so cold that it becomes a cold thing and start to warm back up?

Could the coffee become as cold as the glass of milk? Common sense suggests that can’t happen. When we say coffee grows cold, we mean that it becomes relatively colder, closer to room temperature. And when we say the milk is getting warm, we also mean it is getting closer to room temperature. We’ve never left a hot cup of coffee on a table and come back later to find that it has cooled off so much that it is colder than room temperature. But could there be small fluctuations?

As the coffee and milk head toward room temperature, could they overshoot the target, just by a little bit? Say room temperature is 70 °F, the coffee starts out at 150 °F, and the milk starts out at 40 °F. We don’t expect the coffee to cool down to 40 °F or the milk to warm up to 150 °F. But could the coffee cool down to 69.5 °F and then go back up to 70 °F? Could the milk warm up to 70.5 °F and then cool back down to 70 °F?

I didn’t get a satisfactory answer to my childhood question until I was in college. Then I found out about Newton’s law of cooling. It says that the rate at which a warm body cools is proportional to the difference between its current temperature and the ambient temperature. This law can be written as a differential equation whose solution shows that the temperature of a warm body decreases exponentially to the ambient temperature. The temperature curve always slopes downward. It doesn’t wiggle even a little on its journey to room temperature. Cold bodies warm up the opposite way, exponentially approaching room temperature but never exceeding it.

In case this seems obvious, think about thermostats. They don’t work this way. Say the temperature in a room is 85 °F and you’d like it to be 72 °F, so you turn on the air conditioning. Will the temperature steadily lower to 72 °F? Not exactly. If you were to plot the temperature in the room over time and look at the graph from far enough away, it would look like it is steadily going down to the desired temperature. But if you look at the graph more closely, you’ll see wiggles. The AC may cool the room to a little below 72 °F, maybe to 70 °F. The AC would cut off and the temperature would rise to 72 °F. Unlike the cup of hot coffee, the AC will often overshoot its target, though not by too much. The temperature may feel constant, but it is not. It oscillates around the desired temperature.