Daily tips update

RegexTip, a Twitter account for learning regular expressions, starts over today with basics and will progress to more advanced properties over time.

SansMouse, an account for Windows keyboard shortcuts, started over with basics two weeks ago.

Both RegexTip and SansMouse are in a loop, progressing from most basic to more advanced features. (Or perhaps I should say progressing from most familiar to less familiar. Calling some features “basic” and others “advanced” isn’t quite right, especially for keyboard shortcuts.)

The other daily tip accounts don’t post in any particular sequence. I try to alternate elementary and advanced content to some extent, but other than that there’s no order.

Six weeks ago I started two new accounts: CompSciFact and StatFact. In a few days CompSciFact will be the most popular of the daily tip accounts if the current trend continues.

Here are all the accounts:

SansMouse iconRegexTip iconTeXtip iconProbFact iconStatFact iconAlgebraFact iconTopologyFact iconAnalysisFact iconCompSciFact icon

I use Hoot Suite to schedule these accounts. I use the paid version because I have too many accounts for the free version and because the paid version has an API that lets me upload files to schedule tips in bulk. (Hoot Suite has an affiliate program, so I make a little money if you sign up through this link.)

If you have suggestions for tweets, please contact me.

Scientific results fading over time

A recent article in The New Yorker gives numerous examples of scientific results fading over time. Effects that were large when first measured become smaller in subsequent studies. Firmly established facts become doubtful. It’s as if scientific laws are being gradually repealed. This phenomena is known as “the decline effect.” The full title of the article is The decline effect and the scientific method.

The article brings together many topics that have been discussed here: regression to the mean, publication bias, scientific fashion, etc. Here’s a little sample.

“… when I submitted these null results I had difficulty getting them published. The journals only wanted confirming data. It was too exciting an idea to disprove, at least back then.” … After a new paradigm is proposed, the peer-review process is tilted toward positive results. But then, after a few years, the academic incentives shift—the paradigm has become entrenched—so that the most notable results are now those that disprove the theory.

This excerpt happens to be talking about “fluctuating asymmetry,” the idea that animals prefer more symmetric mates because symmetry is a proxy for good genes. (I edited out references to fluctuating asymmetry from the quote to emphasize that the remarks could equally apply to any number of topics. ) Fluctuating asymmetry was initially confirmed by numerous studies, but then the tide shifted and more studies failed to find the effect.

When such a shift happens, it would be reassuring to believe that the initial studies were simply wrong and that the new studies are right. But both the positive and negative results confirmed the prevailing view at the time they were published. There’s no reason to believe the latter studies are necessarily more reliable.

Related posts:

Your job is trivial. (But I couldn’t do it.)

Ever had a conversation that could be summarized like this?

Your job is trivial. (But I can’t do it.)

This happens in every profession. Everyone’s job has difficulties that outsiders dismiss. I’ve seen it in everything I’ve done, but especially in software development. Here are some posts along those lines.

How long computer operations take

The following table is from Peter Norvig’s essay Teach Yourself Programming in Ten Years. All times are in units of nanoseconds.

execute typical instruction1
fetch from L1 cache memory0.5
branch misprediction5
fetch from L2 cache memory7
Mutex lock/unlock25
fetch from main memory100
send 2K bytes over 1Gbps network20,000
read 1MB sequentially from memory250,000
fetch from new disk location (seek)8,000,000
read 1MB sequentially from disk20,000,000
send packet US to Europe and back150,000,000


Occam’s razor and Bayes’ theorem

Occam’s razor says that if two models fit equally well, the simpler model is likely to be a better description of reality. Why should that be?

A paper by Jim Berger suggests a Bayesian justification of Occam’s razor: simpler hypotheses have higher posterior probabilities when they fit well.

A simple model makes sharper predictions than a more complex model. For example, consider fitting a linear model and a cubic model. The cubic model is more general and fits more data. The linear model is more restrictive and hence easier to falsify. But when the linear and cubic models both fit, Bayes’ theorem “rewards” the linear model for making a bolder prediction. See Berger’s paper for a details and examples.

From the conclusion of the paper:

Ockham’s razor, far from being merely an ad hoc principle, can under many practical situations in science be justified as a consequence of Bayesian inference. Bayesian analysis can shed new light on what the notion of “simplest” hypothesis consistent with the data actually means.

Click to learn more about Bayesian statistics consulting


Related links:

Demand for simplicity?

From Donald Norman’s latest book Living with Complexity:

… the so-called demand for simplicity is a myth whose time has passed, if it ever existed.

Make it simple and people won’t buy. Given a choice, they will take the item that does more. Features win over simplicity, even when people realize that features mean more complexity. You do too, I’ll bet. Haven’t you ever compared two products side by side, feature by feature, and preferred the one that did more? …

Would you pay more money for a washing machine with fewer controls? In the abstract, maybe. At the store, probably not.

Donald Norman’s assessment sounds wrong at first. Don’t we all like things to be simple? Not if by “simple” we mean “fewer features.”

A general theme in Living with Complexity is that complexity is inevitable and often desirable, but it can be managed. We say we want things that are simple, but we really want things that are easy to use. The book gives several examples to illustrate how different those two ideas are.

If something is complex but familiar and well designed, it’s easy to use. If something is simple but unfamiliar or poorly designed, it’s hard to use.

Related posts:

Some programmers really are 10x more productive

One of the most popular post on this site is Why programmers are not paid in proportion to their productivity. In that post I mention that it’s not uncommon to find some programmers who are ten times more productive than others. Some of the comments discussed whether there was academic research in support of that claim.

I’ve seen programmers who were easily 10x more productive than their peers. I imagine most people who have worked long enough can say the same. I find it odd to ask for academic support for something so obvious. Yes, you’ve seen it in the real world, but has it been confirmed in an artificial, academic environment?

Still, some things are commonly known that aren’t so. Is the 10x productivity difference exaggerated folklore? Steve McConnell has written an article reviewing the research behind this claim: Origins of 10x — How valid is the underlying research?. He concludes

The body of research that supports the 10x claim is as solid as any research that’s been done in software engineering.

Related posts:

Another math calendar from Ron Doerfler

Last year Ron Doerfler made a beautiful calendar with images from graphical computing, charts used as computational aids before desktop calculators were ubiquitous.

Ron has made a new calendar and this year’s theme is lightning computing, tricks for mental calculation. The calendar is available for download as a PDF or for purchase in hard copy.

first pages from calendar

April calendar

Related post:

Three views of differential equations

The most common view of differential equations may be sheer terror, but those who get past terror may have one of the following perspectives.

Naive view: All differential equations can be solved in closed form by applying one of the 23 tricks covered in your text book.

Sophomoric view: Differential equations that come up in practice can almost never be solved in closed form, so it’s not worth trying. Learn numerical techniques and don’t bother with analytic solutions.

Classical view: Some very important differential equations can be solved in closed form, especially if you expand your definition of “closed form” to include a few special functions.  Analytic solutions to these equations will tell you things that would be hard to discover from numerical solutions alone.

* * *

I never held the naive view; I learned the sophomoric view before I knew much about differential equations. There’s a lot of truth in the sophomoric view — that’s why it’s called sophomoric.  It’s not entirely wrong, it’s just incomplete. (More on that below.)

I’ve learned differential equations in a sort of reverse-chronological order. I learned the modern theory first — existence and uniqueness theorems, numerical techniques, etc. — and only learned the classical theory much later. I studied nonlinear PDEs before knowing much about linear PDEs.  This may be the most efficient way to learn, begin with the end in mind and all that. It almost certainly is the fastest way to get out of graduate school. But it’s not very satisfying.

* * *

I get in trouble whenever I mention etymologies. So at the risk of sounding like Gus Portokalos from My Big Fat Greek Wedding, I’ll venture another etymology. I’ve always heard that sophomore comes from the Greek words sophos (wise) and moros (fool), though something I read suggested this may be a folk etymology. It doesn’t matter: regardless of whether that is the correct historical origin of the word, it accurately conveys the sense of the word. The idea is that a sophomore has learned a little knowledge but is over-confident in that knowledge and doesn’t know its boundaries. In mathematical terms, it’s someone who has learned a first-order approximation to the truth and extrapolates that approximation too far.

* * *

Related posts:


Classical Greek dramatists believed that it was degrading to show extreme emotion on stage. Some action had to be implied off stage (ob skene) because it was unfit to display explicitly. The classical idea of obscenity included sexual conduct, but would also include expressions of anguish.

I’m more concerned about obscenity in the classical sense than the more narrow contemporary sense. I am not disturbed by salty language or innuendo as much as I am by seeing lives turned inside-out publicly. I am deeply offended, for example,  by a reporter shoving a microphone in a hysterical woman’s face and asking her how she feels now that she has lost her husband. That is obscene.

Related post: Place, privacy, and dignity

Three P’s and three I’s of economics

In the December 27 episode of EconTalk, Pete Boettke summarizes basic economics as follows: If you don’t have the three P’s, you can’t have the three I’s.

The three P’s are

  • Property
  • Prices
  • Profit and loss

The three I’s are

  • Information
  • Incentive
  • Innovation

Related posts:

Style and understanding

From Let Over Lambda by Doug Hoyte:

Style is necessary only when understanding is missing. A corollary to this is that sometimes the only way to effectively use something you don’t understand is to copy styles observed elsewhere.

I liked those lines when I first read them. But as I thought about them more, they started to sound sophomoric.

In context, Hoyte  is arguing that one should not avoid advanced programming techniques just because they are not in common use. Also, I believe he has in mind a single programmer working in isolation. Hoyte’s statement is easier to accept within those boundaries than when applied more generally, but even in context there is room to disagree.

Novices may not realize that a style is a style. They may confuse what they find necessary with what is necessary.

But style can be the mark of experts as well as novices. Novices may follow a convention because they know no alternative. Experts may be aware of alternatives and deliberately choose the limitations of the same convention.Experts may see the wisdom in convention, or may see convention as a small price to pay out of consideration for other people.

It’s not saying much to say style is only necessary “when understanding is missing.” Understanding is nearly always missing to some extent on any large project. We hardly ever understand what we’re doing so thoroughly that we can completely disregard style.

Related posts:

Top five non-technical posts of 2010

Most of last year’s most popular posts here were about math and programming. Here are the most popular posts from 2010 not about math or programming. (They may mention math or programming, but they’re not about math or programming.)

A couple of these posts were written in 2009 but got a lot of traffic in 2010.

I almost included the following post in the list. It’s somewhat about math, but it’s more about life in general: Mathematically correct buy psychologically wrong.