Haskell is to math as Perl is to English?

Fortran superficially looks like mathematics. Its name comes from “FORmula TRANslation,” and the language does provide a fairly straight-forward way to turn formulas into code. But the similarity to mathematics ends there at the lowest level of abstraction.

Haskell, on the other hand, is more deeply mathematical. Fortran resembles math in the small, but Haskell resembles math in the large. Haskell has mathematical structures at a higher level of abstraction than just formulas. As a recent article put it

While Fortran provides a comfortable translation of mathematical formulas, Haskell code begins to resemble mathematics itself.

On its surface, Perl is one of the least English-like programming languages. It is often criticized as looking like “line noise” because of its heavy use of operators. By contrast, Python has been called “executable pseudocode” because the source is easier to read, particularly for novice programmers. And yet at a deeper level, Perl is more English-like than other programming languages such as Python.

Larry Wall explains in Natural Language Principles in Perl that he designed Perl to incorporate features of spoken language. For example, Perl has analogs of articles and pronouns. (Larry Wall studied both linguistics and computer science in college.) Opinions differ on how well his experiment with incorporating natural language features into programming languages has worked out, but it was an interesting idea.

Related posts:

Normal approximation details

The normal distribution can approximate many other distributions, though the details such as quantitative error estimates and what factors improve or degrade the approximation are harder to find. Here are some notes on normal approximations to several common probability distributions.

* * *

For daily posts on probability, follow @ProbFact on Twitter.

ProbFact twitter icon

New trade-offs

Technology is all about trade-offs.

I find it more plausible when someone says a new technology has new trade-offs than when someone says a new technology is “better.” Rarely does one thing improve on another by all criteria, but often one thing is an improvement on another by the criteria you value most.

If you want to persuade me to adopt something new, you’ll gain credibility by being candid about its drawbacks. Explain by what criteria you think the new thing is better, by what criteria it is worse, and why the former should matter more to me in my circumstances.

Sometimes definitions are enough

Sometimes you can apply math just by raiding it for vocabulary. You may not need to apply a single theorem.

This has been a surprise to me. I’m more used to creating a mathematical model so you can compute something or apply some theorem. But sometimes you can move a project along just by providing a name for a concept. A meandering discussion can snap into focus because someone has a name for an idea everyone vaguely understands.

Sometimes it may be clear that only part of a mathematical definition applies. In this case math can guide the discussion by asking whether the rest of the definition applies. “It sounds like we’ve got a widget here. A widget has to have these five properties and clearly we have the first three. Let’s think about whether the last two hold.” The answers don’t have to be positive to be useful. You might realize something important in the process of explaining why your thing is not a widget.

Sometimes a definition may not apply at all and still be useful! “This reminds me of a widget. It’s not a widget in any strict sense. But if it were, this is what we’d do next. I wonder whether we can do something like that.”

Robust in one sense, sensitive in another

When you sort data and look at which sample falls in a particular position, that’s called order statistics. For example, you might want to know the smallest, largest, or middle value.

Order statistics are robust in a sense. The median of a sample, for example, is a very robust measure of central tendency. If Bill Gates walks into a room with a large number of people, the mean wealth jumps tremendously but the median hardly budges.

But order statistics are not robust in this sense: the identity of the sample in any given position can be very sensitive to perturbation. Suppose a room has an odd number of people so that someone has the median wealth. When Bill Gates and Warren Buffett walk into the room later, the value of the median income may not change much, but the person corresponding to that income will change.

One way to evaluate machine learning algorithms is by how often they pick the right winner in some sense. For example, dose-finding algorithms are often evaluated on how often they pick the best dose from a set of doses being tested. This can be a terrible criteria, causing researchers to be mislead by a particular set of simulation scenarios. It’s more important how often an algorithm makes a good choice than how often it makes the best choice.

Suppose five drugs are being tested. Two are nearly equally effective, and three are much less effective. A good experimental design will lead to picking one of the two good drugs most of the time. But if the best drug is only slightly better than the next best, it’s too much to expect any design to pick the best drug with high probability. In this case it’s better to measure the expected utility of a decision rather than how often a design makes the best decision.

Haskell / Category theory decoder ring

Haskell uses a lot of ideas from category theory, but the correspondence between Haskell and category theory can be a little hard to see at times.

One difficulty is that although Haskell articles use terms like functor and monad from category theory, they seldom actually talk about categories per se. If we’ve got functors, where are the categories? (This reminds me of Darth Vader asking “If this is a consular ship, where is the ambassador?”)

In Haskell literature, everything implicitly lives Hask, the category of Haskell types, or in some subcategory of Hask. This means that the category itself is not the focus of attention. In category theory, functors often operate between very different classes of objects, such as topological spaces and their fundamental groups, and so it’s more important to state what category something lives in.

Another potential stumbling block is to think of Haskell types as categories and values as objects. That would be reasonable, since in computer science an “object” is an instance of a type. But the right correspondence is to think of Haskell types as categorical objects. Instances of types are below the level of abstraction we’re working at. This is analogous to how category theory treats objects as black boxes with no way to talk about what’s inside.

Finally, Haskell monads look a little different from categorical monads. Haskell’s return corresponds directly to unit, usually written as η, in category theory. But Haskell monads have a bind operator >>= while mathematical monads have a join operator μ. These are not equivalent, though you can implement each in terms of the other:

join :: Monad m => m (m a) -> m a
join x = x >>= id

(>>=) :: Monad m => m a -> (a -> m b) -> m b
x >>= f = join (fmap f x)

To read more along these lines, see the Wikibooks article on Haskell and Category theory.

Update: Stephen Diehl suggested I mention the differences between the idealized category Hask and the implementation of the Haskell language. These are discussed here.

Related: Applied category theory

An algebra and a calculus

One generation develops an area of math and a second generation comes along to tidy up the definitions. Sometimes this leads to odd vocabulary because the new generation wants to retain terms even as it gives them different meanings.

In high school you get used to hearing about “algebra” and “calculus” as subjects, then the first time you hear someone say “an algebra” or “a calculus” it sounds bizarre. Similarly, once you’re used to hearing about logic, statistics, or topology, it sounds exotic to hear someone say “a logic,” “a statistic,” or “a topology,” something like hearing “a speed of light.”

When I signed up for a topology course, visions of “rubber sheet geometry” danced in my head. I was taken aback when the course started out by defining a topology to be a collection of sets closed under finite intersections and arbitrary unions. Where did the rubber sheet go?

Sometimes when a subject takes on an indefinite article, the new definition comes early in the development. For example, “a statistic” is any function of a random sample, a disappointing definition that comes early in a statistics course. But sometimes the definition with the indefinite article comes much later.

You can take a couple courses in algebra in high school and even a college course in algebra (i.e. groups, rings, and fields) without ever hearing the definition of “an algebra.” And you can take several courses in calculus without ever hearing of “a calculus.”

The phrase “an algebra” can have several meanings, and the phrase “a calculus” even more. The core definition of an algebra is a vector space with a bilinear product. But there are more general ideas of an algebra, some so general that they mean “things and operations on things.” (That’s basically universal algebra.)

Often you put an adjective in front of algebra to specify what kind of algebra you’re talking about: Banach algebra, σ-algebra, Boolean algebra, etc. Usually in mathematics, an adjective in front of a noun narrows the definition. For example, an Abelian group is a particular kind of group. But often an algebra with a qualifier, such as a σ-algebra, is not an algebra by what I called the core definition above.

The idea of “a calculus” is even more vague. If it has a formal definition, I’ve never seen it. A calculus is just a subject with a set of useful rules for calculating things. Sometimes there’s a strong analogy to calculus in the sense of differential calculus, as in the calculus of finite differences, a.k.a. umbral calculus. Sometimes the analogy is more of a stretch, as in λ calculus or π calculus.

So what is the point of this post? I don’t really have one. Just throwing out some half-baked musings.

The secret to planting St. Augustine sod

I’ve put down St. Augustine grass numerous times and it never did well until this year. I’d water it regularly all summer and yet most of it would die.

When I ordered a pallet of grass a few weeks ago, the farmer who grew the grass delivered it. He told me the trick was to water it heavily every day for one week, then water whenever you’d water the rest of your grass. For the first week, you want to water it so much that when you step on it you see muddy water bubble up. This breaks down the sticky clay soil that the grass comes in so that it will put roots down into the soil beneath. So far it looks like his advice worked.


St. Augustine grass

Paper and pixels

This morning a friend came up to me and said “I really liked that article you linked to the other day, though I can’t remember what it was about.”

He said something else that made me think which one he might have meant. “Was it that article that says we don’t remember what we read online as well as what we read on paper?”

“Yeah! That was it!”

Look-behind regex

Look-behind is one of those advanced/obscure regular expression features that I don’t use frequently enough to remember the syntax, but just frequently enough that I wish I could remember it.

Look-behind can be positive or negative. Look-behind says “match this position only if the preceding text matches (does not match) the following  pattern.”

The syntax in Perl and similar regular expression implementations is (?<= … ) for positive look-behind and (?<! … ) for negative look-behind. For the longest time I couldn’t remember whether the next symbol after ? was the direction (i.e. < for behind) or the polarity (= for positive, ! for negative). I was more likely to guess wrong unless I’d used the syntax recently.

The reason I was tempted to get these wrong is that I thought “positive look-behind” and “negative look-behind.” That’s how these patterns are described. But this means the words and symbols come in a different order. If you think look-behind positive and look-behind negative then the words and the symbols come in the same order:


Maybe this syntax comes more naturally to people who speak French and other languages where adjectives follow the thing they describe. English word order was tripping me up.

By the way, the syntax for look-ahead patterns is simpler: just leave out the <. The default direction for look-around patterns is forward. You don’t have to remember whether the symbol for direction or parity comes first because there is no symbol for direction.

* * *

For daily tips on regular expressions, follow @RegexTip on Twitter.

Regex tip icon