A couple days ago I wrote a blog post quoting Cal Newport suggesting that four hours of intense concentration a day is as much as anyone can sustain. That post struck a chord and has gotten a lot of buzz on Hacker News and Twitter. Most of the feedback has been agreement, but a few people have complained that this four-hour limit is based only on anecdotes, not scientific data.
Realistic scientific studies of productivity are often not feasible. For example, people often claim that programming language X makes them more productive than language Y. How could you conduct a study where you randomly assign someone a programming language to use for a career? You could do some rinky-dink study where you have 30 CS students do an artificial assignment using language X and 30 using Y. But that’s not the same thing, not by a long shot.
If someone, for example Rich Hickey, says that he’s three times more productive using one language than another, you can’t test that assertion scientifically. But what you can do is ask whether you think you are similar to that person and whether you work on similar problems. If so, maybe you should give their recommended language a try.
Suppose you wanted to test whether people are more productive when they concentrate intensely for two hours in the morning and two hours in the afternoon. You couldn’t just randomize people to such a schedule. That would be like randomizing some people to run a four-minute mile. Many people are not capable of such concentration. They either lack the mental stamina or the opportunity to choose how they work. So you’d have to start with people who have the stamina and opportunity to work the way you want to test. Then you’d randomize some of these people to working longer, fractured work days. Is that even possible? How would you keep people from concentrating? Harrison Bergeron anyone? And if it’s possible, would it be ethical?
Real anecdotal evidence is sometimes more valuable than artificial scientific data. As Tukey said, it’s better to solve the right problem the wrong way than to solve the wrong problem the right way.
You can test individuals skills rigorously, and many of these effects should appear on short time scales as well as long time scales. For example, working relatively short time periods is strikingly analogous to the central observation of spaced repetition: that short periods of studying spread over a long time beats the same total period of studying spread over a short time, by possibly very large amounts. This might sound difficult to impossible to test: ‘how are you going to assign people to study the *same* stuff according to your arbitrary schedule and measure their recall over *years*?’ But the effect can be shown over periods of seconds, minutes, days, weeks, and months; so with all that background, it’s much more credible that the effect will extend out to years – and the handful of studies to test the effect over such long intervals do find it there as well.
Similarly for productivity. Yes, if this is true, why can’t we observe it over short time periods? And shouldn’t we be able to observe it in other intellectual activities? I’ve already mentioned spaced repetition, but apparently there are similar studies saying that lectures should not go more than 20 minutes and classes run into quickly diminishing returns.
We could get hints from more feasible studies. The results would be interesting and useful. But there’s still great potential to be misled. Sometimes short-term performance is poorly correlated or even negatively correlated with long-term performance. The factors that make a programmer productive, for example, show up on the time scale of months and years, not hours and days.
Studying and producing are opposite activities in a sense. Studying trying to put something into your brain. Productivity is trying to pull something out of your brain. Spaced repetition is about repeating something known, not discovering something new. Short, frequent periods of study are fine for memorizing French vocabulary, but not for inventing algorithms.
The problem is not anecdotes, but cherry-picked anecdotes. In that previous post, you link to commenters reporting that some people claim to work in short bursts, but not to commenters reporting that other people claim to maintain productivity in long periods. You say that people brag about long hours, but have no concern that people may brag about peak productivity.
Wouldn’t this be a problem best suited by problem inversion like genetic screening? You can’t control what populations you sample but you can take lots of samples and bin them based on success and failure. Then you look to see if there is a disproportionate representation of “success” in the subset of your sample space that happens to fall into the “2 hours of concentration” bucket.
The field of econometrics is essentially designed to deal with situations like this, where there are a lot of confounding effects and self-selection going on. Techniques for getting around this include regression discontinuity, instrumental variables, and propensity matching methods. Some work better than others.
(I’m not an economist, but I have been interviewing lots of them lately.)
I have to quibble with Tukey, though. Addressing the right problem the wrong way can be every bit as bad as addressing the wrong problem. I prefer the quotation “It’ s better to be approximately right than exactly wrong.”
On the question of correlation between short (e.g. tests) and long (e.g. career) performance, a comment in the thread at http://languagelog.ldc.upenn.edu/nll/?p=4461 could be interesting. Search the comments for the word
“dopamine”. In short there are two variants of a gene related to the clearing of dopamine. That article, like many on Language Log discusses statistics and “Scientific Journalism”. And how they can (mis-) guide policy.
Didier Sornette has written about his regimen to achieve 10–14 hours of productive physics work daily (he runs in some of the same circles as Nassim Taleb): http://arxiv.org/ftp/arxiv/papers/1111/1111.4621.pdf since he’s a Frenchman and a physicist, he naturally focuses on (i) sleep, (ii) sex, (iii) breath and rigorous but brief exercise (recall that deep breathing is a workout for your abs), (iv) water and chewing (absolutely no engineered drinks), (v) nearly a vegan diet (here he diverges from Taleb and de Vany, who are omnivorous, but I like Sornette’s attitude of “try it for yourself”), (vi) power foods, and (vii) exploitation of the relationship between play and will. I will try (and fail) to avoid making any comments about the “mee too”-ism of members of the internet community in justifying their inadequate interest in optimizing their entire environment for productivity.