Your job is trivial. (But I couldn’t do it.)

Ever had a conversation that could be summarized like this?

Your job is trivial. (But I can’t do it.)

This happens in every profession. Everyone’s job has difficulties that outsiders dismiss. I’ve seen it in everything I’ve done, but especially in software development. Here are some posts along those lines.

How long computer operations take

The following table is from Peter Norvig’s essay Teach Yourself Programming in Ten Years. All times are in units of nanoseconds.

execute typical instruction1
fetch from L1 cache memory0.5
branch misprediction5
fetch from L2 cache memory7
Mutex lock/unlock25
fetch from main memory100
send 2K bytes over 1Gbps network20,000
read 1MB sequentially from memory250,000
fetch from new disk location (seek)8,000,000
read 1MB sequentially from disk20,000,000
send packet US to Europe and back150,000,000

 

Occam’s razor and Bayes’ theorem

Occam’s razor says that if two models fit equally well, the simpler model is likely to be a better description of reality. Why should that be?

A paper by Jim Berger suggests a Bayesian justification of Occam’s razor: simpler hypotheses have higher posterior probabilities when they fit well.

A simple model makes sharper predictions than a more complex model. For example, consider fitting a linear model and a cubic model. The cubic model is more general and fits more data. The linear model is more restrictive and hence easier to falsify. But when the linear and cubic models both fit, Bayes’ theorem “rewards” the linear model for making a bolder prediction. See Berger’s paper for a details and examples.

From the conclusion of the paper:

Ockham’s razor, far from being merely an ad hoc principle, can under many practical situations in science be justified as a consequence of Bayesian inference. Bayesian analysis can shed new light on what the notion of “simplest” hypothesis consistent with the data actually means.

 

Click to learn more about Bayesian statistics consulting

 

Related links:

Demand for simplicity?

From Donald Norman’s latest book Living with Complexity:

… the so-called demand for simplicity is a myth whose time has passed, if it ever existed.

Make it simple and people won’t buy. Given a choice, they will take the item that does more. Features win over simplicity, even when people realize that features mean more complexity. You do too, I’ll bet. Haven’t you ever compared two products side by side, feature by feature, and preferred the one that did more? …

Would you pay more money for a washing machine with fewer controls? In the abstract, maybe. At the store, probably not.

Donald Norman’s assessment sounds wrong at first. Don’t we all like things to be simple? Not if by “simple” we mean “fewer features.”

A general theme in Living with Complexity is that complexity is inevitable and often desirable, but it can be managed. We say we want things that are simple, but we really want things that are easy to use. The book gives several examples to illustrate how different those two ideas are.

If something is complex but familiar and well designed, it’s easy to use. If something is simple but unfamiliar or poorly designed, it’s hard to use.

Related posts:

Some programmers really are 10x more productive

One of the most popular post on this site is Why programmers are not paid in proportion to their productivity. In that post I mention that it’s not uncommon to find some programmers who are ten times more productive than others. Some of the comments discussed whether there was academic research in support of that claim.

I’ve seen programmers who were easily 10x more productive than their peers. I imagine most people who have worked long enough can say the same. I find it odd to ask for academic support for something so obvious. Yes, you’ve seen it in the real world, but has it been confirmed in an artificial, academic environment?

Still, some things are commonly known that aren’t so. Is the 10x productivity difference exaggerated folklore? Steve McConnell has written an article reviewing the research behind this claim: Origins of 10x — How valid is the underlying research?. He concludes

The body of research that supports the 10x claim is as solid as any research that’s been done in software engineering.

Related posts:

Another math calendar from Ron Doerfler

Last year Ron Doerfler made a beautiful calendar with images from graphical computing, charts used as computational aids before desktop calculators were ubiquitous.

Ron has made a new calendar and this year’s theme is lightning computing, tricks for mental calculation. The calendar is available for download as a PDF or for purchase in hard copy.

first pages from calendar

April calendar

Related post:

Three views of differential equations

The most common view of differential equations may be sheer terror, but those who get past terror may have one of the following perspectives.

Naive view: All differential equations can be solved in closed form by applying one of the 23 tricks covered in your text book.

Sophomoric view: Differential equations that come up in practice can almost never be solved in closed form, so it’s not worth trying. Learn numerical techniques and don’t bother with analytic solutions.

Classical view: Some very important differential equations can be solved in closed form, especially if you expand your definition of “closed form” to include a few special functions.  Analytic solutions to these equations will tell you things that would be hard to discover from numerical solutions alone.

* * *

I never held the naive view; I learned the sophomoric view before I knew much about differential equations. There’s a lot of truth in the sophomoric view — that’s why it’s called sophomoric.  It’s not entirely wrong, it’s just incomplete. (More on that below.)

I’ve learned differential equations in a sort of reverse-chronological order. I learned the modern theory first — existence and uniqueness theorems, numerical techniques, etc. — and only learned the classical theory much later. I studied nonlinear PDEs before knowing much about linear PDEs.  This may be the most efficient way to learn, begin with the end in mind and all that. It almost certainly is the fastest way to get out of graduate school. But it’s not very satisfying.

* * *

I get in trouble whenever I mention etymologies. So at the risk of sounding like Gus Portokalos from My Big Fat Greek Wedding, I’ll venture another etymology. I’ve always heard that sophomore comes from the Greek words sophos (wise) and moros (fool), though something I read suggested this may be a folk etymology. It doesn’t matter: regardless of whether that is the correct historical origin of the word, it accurately conveys the sense of the word. The idea is that a sophomore has learned a little knowledge but is over-confident in that knowledge and doesn’t know its boundaries. In mathematical terms, it’s someone who has learned a first-order approximation to the truth and extrapolates that approximation too far.

* * *

Related posts:

Obscenity

Classical Greek dramatists believed that it was degrading to show extreme emotion on stage. Some action had to be implied off stage (ob skene) because it was unfit to display explicitly. The classical idea of obscenity included sexual conduct, but would also include expressions of anguish.

I’m more concerned about obscenity in the classical sense than the more narrow contemporary sense. I am not disturbed by salty language or innuendo as much as I am by seeing lives turned inside-out publicly. I am deeply offended, for example,  by a reporter shoving a microphone in a hysterical woman’s face and asking her how she feels now that she has lost her husband. That is obscene.

Related post: Place, privacy, and dignity

Three P’s and three I’s of economics

In the December 27 episode of EconTalk, Pete Boettke summarizes basic economics as follows: If you don’t have the three P’s, you can’t have the three I’s.

The three P’s are

  • Property
  • Prices
  • Profit and loss

The three I’s are

  • Information
  • Incentive
  • Innovation

Related posts:

Style and understanding

From Let Over Lambda by Doug Hoyte:

Style is necessary only when understanding is missing. A corollary to this is that sometimes the only way to effectively use something you don’t understand is to copy styles observed elsewhere.

I liked those lines when I first read them. But as I thought about them more, they started to sound sophomoric.

In context, Hoyte  is arguing that one should not avoid advanced programming techniques just because they are not in common use. Also, I believe he has in mind a single programmer working in isolation. Hoyte’s statement is easier to accept within those boundaries than when applied more generally, but even in context there is room to disagree.

Novices may not realize that a style is a style. They may confuse what they find necessary with what is necessary.

But style can be the mark of experts as well as novices. Novices may follow a convention because they know no alternative. Experts may be aware of alternatives and deliberately choose the limitations of the same convention.Experts may see the wisdom in convention, or may see convention as a small price to pay out of consideration for other people.

It’s not saying much to say style is only necessary “when understanding is missing.” Understanding is nearly always missing to some extent on any large project. We hardly ever understand what we’re doing so thoroughly that we can completely disregard style.

Related posts:

Top five non-technical posts of 2010

Most of last year’s most popular posts here were about math and programming. Here are the most popular posts from 2010 not about math or programming. (They may mention math or programming, but they’re not about math or programming.)

A couple of these posts were written in 2009 but got a lot of traffic in 2010.

I almost included the following post in the list. It’s somewhat about math, but it’s more about life in general: Mathematically correct buy psychologically wrong.

Mathematical landscape

W. W. Sawyer makes a beautiful analogy regarding the mathematical landscape in his book Prelude to Mathematics.

Imagine farmers living in a country where no other tool was available except the wooden plough. Of necessity, the farms would have to be in those places where the earth was soft enough to be cultivated with a wooden implement. If the population grew sufficiently to occupy every suitable spot, the farms would become a map of the soft earth regions. …

It is much the same with mathematical research. At any stage of history, mathematicians possess certain resources of knowledge, experience, and imagination. These resources are sufficient to resolve some problems but not others. … Unconsciously, therefore, the map of mathematical knowledge comes to resemble the map of problems soluble by given tools.

But of course the discoveries themselves open the way for the invention of fresh tools. As the coming of the steel plough would change the map of the farmlands, so these new tools open up new regions of profitable research. But the new tools may take centuries to come, and while we wait for them, the frontier remains an impassable barrier.

Related post: Easy to guess, hard to prove

The solar system in a glass of wine

William Blake’s poem Auguries of Innocence opens with these famous lines:

To see a world in a grain of sand,
And a heaven in a wild flower,
Hold infinity in the palm of your hand,
And eternity in an hour.

This poem came to mind when I saw @mathematicsprof post the following on Twitter:

At your next holiday party, look straight down into your glass of wine and tilt the glass one degree. You will see the elliptic orbit of the earth.

If you tilt your glass 12 degrees you’ll see the orbit of Mercury. In general, if you tilt your glass θ degrees you’ll see an ellipse with eccentricity sin(θ).

(I’ve taken the liberty of editing the original tweets to take advantage of the extra breathing room outside of Twitter. Original tweets here and here.)

I like this for two reasons: it’s a great astronomy illustration, and it’s an example of how much information you can get into two 140-character messages.

Two contrasting articles on minimalism

This morning I ran across a couple articles on minimalism:

The former has a sense of humor; the latter does not. The former contains thoughtful criticism; the latter is a knee-jerk reaction. The former makes an interesting argument; the latter quibbles about definitions.

The former article is by Vivek Haldar. I cannot tell who wrote the latter.

Here’s an excerpt from Haldar’s article:

The zenith … is a calm geek, sitting in a bare room with a desk upon which sits only a MacBook Air, his backpack of possessions on one side, the broadband Internet cable available but unplugged, fingers ready to type into the empty white screen of a minimalist editor.

I think that’s pretty funny. And I would hope that minimalists would be able to get a chuckle out of it.

But Haldar does not just lampoon hipster minimalism. He argues that you need periods of stimulation and clutter to be creative. He also argues that minimalism has its place.

Now I agree with most of the premises of the minimalists … My gripe is with the way they sell it as a way of life. It’s much more valuable as a periodic phase of life.

Minimalism cannot be a long-term strategy, but it makes an excellent short-term tactic.

The second article essentially argues that Haldar has the definition of minimalism wrong.

Minimalism, at its core, is the process of prioritizing your life and working towards concrete goals without giving in to distraction. … Like any school of thought with a certain critical mass, there is dissent and corruption among the ranks.

Who can find fault with prioritizing your life, working toward concrete goals, and avoiding distraction? And who wants to defend corruption? But this is just quibbling about definitions. By contrast, Haldar makes an argument independent of such a definition. Haldar argues that a certain set of attitudes and behaviors — however you want to label them — are not conducive to sustained creativity.

Here are some ideas I threw out a while ago on defining minimalism.

“Minimal” literally means an extreme. I appreciate moderate minimalists, though strictly speaking “moderate minimalist” is a contradiction in terms. A more accurate but unwieldy name for minimalists might be “people who are keenly aware of the indirect costs of owning stuff.”

… you could define a minimalist as someone who wants to eliminate non-essential possessions … But by that definition, Donald Trump would be a minimalist if he believes everything he owns is essential.

Generic discussions of minimalism are fluff. Haldar’s argument is more substantial because he makes a specific suggestion.

Related posts: