Enjoyment of one’s tools

Here’s a quote from Donald Knuth I’ve been thinking about lately:

The enjoyment of one’s tools is an essential ingredient of successful work.

That makes a lot of sense. So does a quote from Joe Armstrong that I blogged about a few weeks ago:

Forget about the tools …

I don’t think Knuth and Armstrong necessarily disagree. I’ll explain the context of each quote and how I think they fit together.

The context of Knuth’s quote is a discussion of floating point arithmetic. It’s the last sentence of Section 4.2.2A of Seminumerical Algorithms. He’s saying that even though floating point arithmetic is inherently inexact, it’s important that it still has certain nice mathematical properties. I believe he intended the quote to be taken more broadly, in part because he italicized it. In any case, his immediate idea of a “tool” was very low-level.

The context of Armstrong’s quote is his advice to stay away from unnecessary software tools, especially when first learning a language. He says that

IDEs and build tools are the single biggest obstacle I know of to getting started.

He goes on to say that his basic programming set up is a shell, makefiles, and Emacs. These are tools, and I imagine Armstrong enjoys using them. They’re high-level tools compared to floating point arithmetic, but low-level compared to IDEs and build tools. (Incidentally, Knuth also uses Emacs.)

So, one way to synthesize the advice from Knuth and Armstrong, along with my personal opinion, would be this:

Pick a small set of tools that you enjoy using and learn them well.

Or possibly this:

Pick a small set of good tools and learn them so well that you enjoy using them.

Related post: Doing good work with bad tools

Math superhero in training

Steve Yegge has a new project. He’s in training to become a math superhero. Or at least a sidekick. He said that math/stat folks superheros and he wants to join them.

In his presentation at OSCON Data 2011 on Monday, Yegge said that all the hard problems require math and statistics. So he’s quitting his job at Google to study math in hopes that he can solve big problems three to five years from now.

His enthusiasm for math is naive and inspiring. I gather from some of the articles he’s written that he’s an original thinker and a hard worker. It’ll be interesting to see what he does.

Update: As pointed out in the comments, Steve Yegge clarified on his blog that he is not quitting Google, only his project at Google.

A chip off the old fractal block

You can make a fractal by starting with a cube and recursively removing chunks of it. The result is called a Menger sponge. Here’s a picture after four levels of recursion.

Menger sponge photo via Wikipedia

What would it look like if you sliced this open? If you cut parallel to one of the faces it looks just like the original object. But if you cut at an odd angle, you’d be surprised. Here’s a New York Times article about the result.

Karl Menger came up with his sponge in 1926, but only recently did someone think to slice one open as in the article.

How to get started with functional programming

Someone asked me this weekend how to get started with functional programming. My answer: Start by writing pure functions in the programming language you’re currently using.

The only input to a pure function is its argument list and the only output is its return value. If you haven’t seen this before, you might think all functions are pure. After all, any function takes in values and returns a value. But in conventional programming there are typically out-of-band ways for information to flow in or out of a function. For example, an impure function may depend on a global variable or class member data. In that case, it’s behavior is not entirely determined by its arguments. Similarly, an impure function might set a global variable or write to a database. In that case the function has a side effect in addition to its return value.

You can write pure functions in any language, though it’s easier in some languages than others. For example, no one would call Fortran a functional language, but there are people (M. J. D. Powell comes to mind) who discipline themselves to write pure functions in Fortran.

Why write pure functions? A pure function has referential transparency, meaning it will always return the same value when given the same inputs. The output does not depend on the system time, the state of a database, which functions were called previously, or anything else that is not explicitly passed as an argument to the function. This means pure functions are easier to understand (and hence easier to debug and test).

You can’t always write pure functions. If you need to stick a value in a database, this cannot be accomplished with a pure function. Or if you’re calling a random number generator, you don’t want it to have referential transparency, always returning the same output! But the goal is to use pure functions when practical. You want to eliminate out-of-band communication when it’s convenient to do so. Total purity is not practical; some argue that the sweet spot is about 85% purity.

So why don’t programmers use pure functions more often? One reason is that pure functions require longer argument lists. In an object oriented language, object methods can have shorter argument lists by implicitly depending on object state. The price to pay for shorter method signatures is that you can’t understand a method by itself. You have to know the state of the object when the method is called. Is it worthwhile to give up referential transparency in order to have shorter method signatures? It depends on your context and your taste, though in my opinion its often worthwhile to use longer function signatures in exchange for more pure functions.

Another reason people give for not writing pure functions is that its too expensive to copy large data structures to pass them into a function. But that’s what pointers are for. You can conceptually pass an object into a function without having to actually make a copy of the object’s bits.

You can also fake purity for the sake of efficiency. For example, Mike Swaim left a comment recently giving an example of how memoization sped up a program by several orders of magnitude. (Memoization is a technique of caching computations. When a function is asked to compute something, it first looks to see whether it has already done the calculation. If so, it returns the cached value. If not, it does the calculation and adds its output to the cache.) A function that uses memoization is not strictly pure — its calculations have a persistent impact on the state of its cache — but such a function can still have referential transparency, always returning the same output given the same input. You could say it’s cheating to call such functions pure, and it is, but if you’re really a stickler about it, all pure functions have side effects.

Related posts:

Software exoskeletons

There’s a major divide between the way scientists and programmers view the software they write.

Scientists see their software as a kind of exoskeleton, an extension of themselves. Think Dr. Octopus. The software may do heavy lifting, but the scientists remain actively involved in its use. The software is a tool, not a self-contained product.

Spiderman versus Dr. Ock

Programmers see their software as something they will hand over to someone else, more like building a robot than an exoskeleton. Programmers believe it’s their job to encapsulate intelligence in software. If users have to depend on programmers after the software is written, the programmers didn’t finish their job.

I work with scientists and programmers, often bridging the gaps between the two cultures. One point of tension is defining when a project is done. To a scientist, the software is done when they get what they want out of it, such as a table of numbers for a paper. Professional programmers give more thought to reproducibility, maintainability, and correctness. Scientists think programmers are anal retentive. Programmers think scientists are cowboys.

Programmers need to understand that sometimes a program really only needs to run once, on one set of input, with expert supervision. Scientists need to understand that prototype code may need a complete rewrite before it can be used in production.

The real tension comes when a piece of research software is suddenly expected to be ready for production. The scientist will say “the code has already been written” and can’t imagine it would take much work, if any, to prepare the software for its new responsibilities. They don’t understand how hard it is for an engineer to turn an exoskeleton into a self-sufficient robot.

Related posts:

Two-finger scrolling on Windows

One of my favorite features of Mac laptops is two-finger scrolling. This lets you scroll down a long document similar to the way the middle wheel does on a Windows mouse.

I mentioned on Twitter this evening that it would be nice if Windows had this feature. Apparently many newer Windows laptops come with this capability.  Todd Brooks replied that my laptop might be able to the two-finger scroll if I upgraded my touchpad driver. He was right. I went to the Microsoft Update site and saw that one of the optional updates was exactly what Todd suggested I install.

So far it seems that the two-finger scrolling on my ThinkPad isn’t as smooth as it is on a MacBook Pro, but it’s still nice to have.

You wanted a banana but you got a gorilla holding the banana

Joe Armstrong, creator of Erlang, on software reusability.

I think the lack of reusability comes in object-oriented languages, not functional languages. Because the problem with object-oriented languages is they’ve got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle.

If you have referentially transparent code, if you have pure functions — all the data comes in its input arguments and everything goes out and leave no state behind — it’s incredibly reusable.

Source: Coders at Work. Emphasis added.

I do most of my work in object-oriented languages and I don’t see that changing any time soon. I’m more interested in functional techniques than functional languages: using pure functions, using functions as arguments and return values, etc. As Joe Armstrong says, such code is easier to reuse. If you want to reuse (or test) a functional banana, you don’t have to set up a stateful gorilla to hold the banana first.

Related posts:

Book review queue

I’ve got a backlog of books to review. I may not write more about all of them here so I’ll at least mention them briefly.

Packt Publishing sent me copies of Python 3 Web Development Beginners Guide and Python Testing Cookbook. The former may not be what you expect from the title. It focuses on self-contained development with minimal dependencies. It’s not a book on popular Python web development frameworks like Django.

I have a couple books I need to review for the MAA. The first is Validated Numerics: A Short Introduction to Rigorous Computations. This one is less general than title implies. It’s a book about interval arithmetic and frankly it’s dry as dust so far.

The second MAA book is The History and Development of Nomography. Nomography is the lost art of graphical computing. A great deal of ingenuity and artistry went into producing graphics that allowed users to solve equations without numerical computation. For beautiful examples, see Dead Reckonings, Ron Doerfler’s web site.

Eric Sink has a new book in press Version Control by Example. I enjoyed reading through one of the early drafts. I haven’t yet read the final version though I plan to soon. This book shows how to use distributed version control, comparing several popular systems. It does more than tell you which buttons to push or APIs to call; it shows how to solve problems that come up in day-to-day software development.

The final book on my review list is The Physics Book by Cliff Pickover. This book is the physics analog of The Math Book that Pickover published a couple years ago. I look forward to reading this one based on my enjoyment of The Math Book.

Related posts:

Scalability and immediate appeal

Paul Graham argues that people take bad jobs for the same reasons they eat bad food. The advantages of both are immediately apparent: convenience and immediate satisfaction. The disadvantages take longer to realize. Bad jobs drag down your soul the way bad food drags down your body.

I first read Graham’s essay You Weren’t Meant to Have a Boss when he wrote it three years ago. I read it again this morning when I saw a link to it on Hacker News. I found his thesis less convincing this time around. But he makes two general points that I think I missed the first time.

  1. Watch out for things that are immediately appealing but harmful in the longer term.
  2. Watch out for being part of someone else’s scalability plans.

The first point is familiar advice, but worth being reminded of. The second point is more subtle.

Companies sell bad food for the same reason they offer bad jobs: it scales. It’s easy to create bland food and bland jobs on a large scale. Fresh food and creative jobs don’t scale so well.

When you choose to eat junk food, you more or less consciously choose convenience or immediate satisfaction over long-term benefit. But it may not be obvious when your range of options has been selected for scalability. For example, few students realize how much the educational system has been designed for the convenience of administrators. Being aware of an organization’s scalability needs can help you interact with it more intelligently.

Related posts:

Sage Beginner’s Guide

I like books. Given a choice, I’d much rather read a book than online documentation. Typically a book speaks with one voice and has been more carefully edited. Unfortunately, it can be hard to find books on specialized software. That’s why I was glad to hear there’s a book on Sage, a project that integrates many Python libraries for mathematical computing into a single context.

Craig Finch’s book Sage Beginner’s Guide provides an easy-to-read overview of Sage. The book is filled with examples. In fact, every topic is introduced by an example. Explanations follow the examples in sections entitled “What just happened?”. Follow-up exercises are provided to solidify the material after the example and explanation.

Someone could begin using Sage without knowing Python. They could think of Sage as an open source Mathematica-like application. But one of the strengths of Sage is that its underlying language is Python. And the Sage Beginner’s Guide has two chapters devoted to the Python language, one basic and one advanced.

Finch’s book is primarily focused on Sage as a whole, not the Python components it integrates. However, it does point out the component libraries that provide certain functionality when either the constituent library conflicts with Sage or can be used to refine Sage functionality.

Sage can be challenging to install. It is not yet directly supported on Windows; the recommended way to use Sage on Windows is to download a Linux virtual machine with Sage installed. I was able to install Sage on Ubuntu but not yet on OS X. However, you can try out Sage without installing it by using Sage online notebooks.

I don’t have as much experience with Sage as with some of its components. As far as I can tell, it’s easy to take your experience from component libraries — say NumPy — and bring it over to Sage. It would be harder to take functionality you discovered while using Sage and use it outside of Sage since, though that is to be expected since part of the value Sage adds is smoothing over the peculiarities of each component library.