Python book recommendations

People sometimes ask me to recommend a book for learning to program or a book on Python. If you want both in one book, i.e. to learn Python as a first programming language, take a look at Allen Downey’s new book Think Python.

If you’re an experience programmer and just want a book on the specifics of Python, I’d recommend David Beazley’s Python Essential Reference.

These two books are at opposite ends of the scale. Downey’s book is a very gentle introduction to programming. He takes the time to explain things that authors often don’t realize need to be explained. Beazley’s book is all about Python per se, not programming. The title says “reference,” and it does make a good reference, but you could sit down and read through the first half of the book. The second half of the book really is more of a reference.

For daily tips on Python and scientific computing, follow @SciPyTip on Twitter.

Scipytip twitter icon

Reluctant experts

Douglas Crockford is probably the most visible critic and advocate of JavaScript. His criticism gives his advocacy credibility, and vice versa.

A few weeks ago, at the end of a presentation he explains how he became “the JavaScript guy.”

I just wanted to say one more thing on a personal note. I never wanted to be the JavaScript guy. … The first time I saw JavaScript in 1995 I thought this is the stupidest thing I’ve ever seen. … Five years later … I started writing about how this language is misunderstood. And I became the JavaScript guy. I’m still not comfortable in that role. It’s still not what I ever wanted to be … It’s likely to continue because the language is becoming more important so it isn’t something I can get away from.

It seems fairly common for someone to become well known for something they stumbled into. Their fame catches them by surprise. Why is this?

One reason may be that it’s hard to plan to be well known. You need to be in the right place at the right time. If you see something taking off and decide to become an expert in it, it may be too late to become famous in that area.

Another reason may be that fame is determined by how others value your work, which could be very different from how you value your work. Something you find only incidentally useful on the way to another goal may turn out to be very valuable to others. This is fairly common, as I discuss here.

Related posts:

Flying to Mars in three days

Richard Campbell brought up an interesting idea in his recent Mars geek out show. Suppose you could travel to Mars accelerating at 1 g for the first half the trip, then decelerating at 1 g for the final half of the trip. Along the way you’d feel a force equal to the force of gravity you’re used to, and you’d get there quickly. How quickly? According to the show, just three days.

To verify this figure, we’ll do a very rough calculation. Accelerating at 1 g for time t covers a distance is g t2/2. Let d be the distance to Mars in meters, T the total of the trip in seconds, and g = 9.8 m/s2. In half the trip you cover half the distance, so 9.8 (T/2)2/2 = d/2. So T = 0.64 √d.

The hard part is picking a value for d. To keep things simple, assume you head straight to Mars, or rather straight toward where Mars will be by the time you get there. (In practice, you’d take more of a curved path.) Next, what do you want to use as your straight-line distance? The distance between Earth and Mars varies between about 55 million km and 400 million km. That gives you a time T between 1.7 and 4.7 days.

We don’t have the technology to accelerate for a day at 1 g. As Richard Campbell points out, spacecraft typically accelerate for maybe 20 minutes and coast for most of their journey. They may also pick up speed by slinging around a planet, but there are no planets between here and Mars.


Here’s another gem from The Peter Principle. The book coins the term “hierarchical exfoliation” to describe how organizations rid themselves of both the least competent and the most competent people.

… in most hierarchies, super-competence is more objectionable than incompetence.

Ordinary incompetence, as we have seen, is no cause for dismissal: it is simply a bar to promotion. Super-competence often leads to dismissal, because it disrupts the hierarchy

Emphasis in the original.

Related post: Intellectual traffic jam

Hard analysis, soft analysis

“Hard analysis” and “soft analysis” are technical terms. They don’t necessarily mean “difficult” and “easy.”

Soft analysis often asks qualitative questions where hard analysis asks more quantitative questions. For example, soft analysis might ask whether a series converges, when hard analysis asks at what rate a series converges. Soft analysis is often more abstract, asking questions about classes of functions, where hard analysis asks questions about specific functions.

The difference between soft and hard analysis reminds me of Freeman Dyson’s classification of mathematicians into birds and frogs:

Some mathematicians are birds, others are frogs. Birds fly high in the air and survey broad vistas of mathematics out to the far horizon. They delight in concepts that unify our thinking and bring together diverse problems from different parts of the landscape. Frogs live in the mud below and see only the flowers that grow nearby. They delight in the details of particular objects, and they solve problems one at a time. I happen to be a frog, but many of my best friends are birds.

I was trained as a bird, but I’ve become more of a frog over time. Of course these categories are not exclusive. Frogs sometimes act as birds and vice versa. The categories of hard and soft analysis are not exclusive either. See Terry Tao’s article on relating hard and soft analysis.

Click to learn more about math and computing consulting

Related post: Impure math


Evaluate people by input or output?

According to The Peter Principle, people in hierarchical organizations tend to be promoted until they reach a position at which they are incompetent. The way managers manage depends on whether they have achieved incompetence or are still on the path to it.

The competence of an employee is determined not by outsiders but by his superior in the hierarchy. If the superior is still at a level of competence, he may evaluate his subordinates in terms of the performance of useful work. … This is to say, he evaluates output.

But if the superior has reached his level of incompetence, he will probably rate his subordinates in terms of institutional values: he will see competence as the behavior that supports the rules, rituals and forms of the status quo. Promptness, neatness, courtesy to superiors, internal paperwork, will be highly regarded. In short, such an official evaluates input.

Emphases in the original.

Much of The Peter Principle is humorous and exaggerated, but the excerpt above is neither. It is simply a description of how organizations work.

Related posts:

Work or rest

According a recent biography of Henri Poincaré,

Poincaré … worked regularly from 10 to 12 in the morning and from 5 till 7 in the late afternoon. He found that working longer seldom achieved anything …

Poincaré made tremendous contributions to math and physics. His two-hour work sessions must have been sprints, working with an intensity that could not be sustained much longer.

I expect most of us would accomplish more if we worked harder when we worked, rested more, and cut out half-work.

Pushing an idea

From The 5 Elements of Effective Thinking:

Calculus may hold a world’s record for how far an idea can be pushed. Leibniz published the first article on calculus in 1684, an essay that was a mere 6 pages long. Newton and Leibniz would surely be astounded to learn that today’s introductory calculus textbook contains over 1,300 pages. A calculus textbook introduces two fundamental ideas, and the remaining 1,294 pages consists of examples, variations, and applications — all arising from following the consequences of just two fundamental idea.

Hiring complementary talent

In their new book, Ed Burger and Mike Starbird relate the following incident from a symposium in honor of Albert Einstein.

One of the speakers told a story about being a young assistant to Einstein. He said that during the job interview with Einstein, he admitted that he didn’t know much about relativity, to which Einstein replied, “That’s OK. I already know about relativity.”

Hunt down bad error messages

My printer is unable to clean 51. It won’t work, and all it says is “Unable to Clean 51.”

Here’s my suggestion for finding such useless error messages in a code review: Write a script to extract all string literals from your source code, then read over the output. The beauty of this approach is that the reviewer sees the text without the context of the surrounding code, just as the user does. Ideally someone who is not a programmer would review the output strings. I would hope someone who ran across “Unable to Clean 51” would flag that as something that doesn’t make sense.

It’s not hard to write a script to pull out string literals. If you’d like, you could use the script that accompanies my Code Project article PowerShell Script for Reviewing Text Shown to Users. That script tries to filter out strings that are not user output, such as file paths. It’s a pretty crude script, attempting to handle source code written in several languages. It will have a few false positives, and a few false negatives, but it works quite well for a short script. (Code Project’s “browse source” tab doesn’t work for PowerShell source. You’ll have to download the code to read it.)

Whenever I recommend this script, I run into a few objections that I’ll address below.

Q: Wouldn’t it be better if programmers put all user output text in string tables rather than putting user output text in the main body of their code?

A: Yes, but that takes more effort, and it’s not how most programmers work. And even if you have a policy to put all output strings in a resource table, you’d need something like this script to enforce the policy.

Q: Wouldn’t it be better to have a real parser extract the strings rather than doing regular expression guesswork?

A: Sure.

Q: Wouldn’t it be better to use a spell checker that’s built into your IDE?

A: No. The purpose of the review is not just to check spelling. You also want to catch grammar errors and unhelpful messages.

Q: Wouldn’t it be better to do a complete code review?

A: Yes and no. Complete code reviews are great for overall code quality. But they take a lot of effort and don’t happen often. A review of just the extracted strings takes far, far less time. Also, viewing the output strings out of context is better for catching unhelpful or ungrammatical messages.

Q: Isn’t this simplistic? Doesn’t every company do something like this?

A: If they do, why do I continually find spelling errors, grammatical errors, and unhelpful messages in the software I use?

PowerShell for Developers

PowerShell was written first and foremost for Windows system administrators, and the benefits to this community are clear. It’s not as clear what developers should make of PowerShell.

Administrators can learn PowerShell as a shell first, and gradually transition from interactive use to scripting. They may learn PowerShell as their first programming language and not even give too much thought to the language per se. But a developer has to ask why and when to use PowerShell rather than another language, such as C#.

Doug Finke’s new book Windows PowerShell for Developers (ISBN 1449322700) is “for developers” in a couple ways. First, the style of the book is geared toward developers. The book is small, less than 200 pages, because the author assumes the readers are experienced Windows developers who want to focus on what PowerShell adds to what they already know. Second, the book focuses on tasks a developer might want to do. Rather than show you how to create a new Active Directory user, as many PowerShell books would, this book covers topics such as

  • code generation
  • static analysis
  • interfacing with C#
  • embedding PowerShell in your application
  • working with XML and JSON
  • interfacing with Excel
  • creating DSLs.

So why should software developers use PowerShell? And what tasks should they do in PowerShell? One answer to the first question, implicit in the book’s examples, is that PowerShell makes it possible to carry out common tasks with little code. Another answer explicitly given in the book is integration.

Given PowerShell’s growing integration with the rest of the Windows platform, as PowerShell grows, so does your application.”

The book is full of examples of what tasks a developer might want to do in PowerShell. The examples I found most interesting were embedding PowerShell to provide a scripting language for your application and creating DSLs in PowerShell.

One pattern in the examples is text munging, whether that text is source code or common data file formats. Another pattern is integration, especially integrating Microsoft technologies. PowerShell is designed to make it possible to solve these kinds of problems with a minimum of ceremony.

I’ll close with a couple reasons why might a developer not want to use PowerShell in my opinion. The first is frequency of use. Although PowerShell can solve many problems with significantly less code than C# would require, you have to learn PowerShell first, and you have to use it frequently enough to remember it. You have to use PowerShell enough to repay the time invested in learning and practicing it.

The second reason is size. The C# language and the Visual Studio IDE were designed for large projects, but scale down fairly well for smaller projects. PowerShell was designed for the command line, but scales fairly well for large scripts. If you use PowerShell and C#, you’ll have to decide at what size you want to switch from one language to the other.

Related posts:

Sociable numbers

A number is called perfect if it is the sum of its proper divisors, i.e. all divisors less than itself. For example, 28 is perfect because 1 + 2 + 4 + 7 + 14 = 28.

Amicable numbers are a sort of generalization of perfect numbers. Two numbers a and b are said to be amicable if a is the sum of b‘s proper divisors and vice versa.

The ancient Greeks knew of only one pair of amicable numbers: 220 and 284. Medieval mathematician Thâbit ibn Kurrah discovered two more pairs: (17296, 18416) and (9363584, 9437056). Leonard Euler (1707–1783) found 58 more pairs. Now over 12 million amicable number pairs have been found.

To generalize things further, start with a number n and compute the sum of its proper divisors, then the sum of the divisors of that number, etc. This sequence of numbers is called the aliquot sequence of n. If this sequence is periodic, n is called a sociable number.

If the aliquot sequence has period 1, n is a perfect number. If the sequence has period 2, n is part of an amicable number pair.

Are there numbers whose aliquot sequence has period 3? Not that we know of. Currently the only aliquot sequence periods that have been demonstrated are 4, 5, 6, 8, 9, and 28.

Related posts:

For daily tweets on algebra and other math, follow @AlgebraFact on Twitter.

AlgebraFact logo

Design for outcomes

Designing a device to save lives is not enough. People may not use it, or may not use it correctly. Or be unable to maintain it. Or …

Link to video. (If you know why the embedded video doesn’t appear in some RSS readers and how to fix it, please let me know.)

I’ve seen analogous problems with statistical methods. People will not necessarily adopt a new statistical method just because it is better. And if they do use it, they may use it wrongly, just like medical devices.

(“Better” in the previous paragraph is a loaded term. Statistical methods are evaluated by many criteria: power, robustness, bias, etc. When someone says his new method is better, he means better by the criteria he cares most about. But even when there is agreement on statistical criteria, a superior statistical method may be rejected for non-statistical reasons.)

Related posts:

Understanding recursion II

People often joke that in order to understand recursion, you must first understand recursion. This is funny as far as it goes, but it illustrates a common misunderstanding. (I’m not saying the people who make the joke don’t understand what they’re doing. They probably do, and are having a little fun with the people who do not.)

Novices often think that recursion means solving a problem in terms of itself, which sounds like an infinite regress. And it might be. But an important qualifier is missing in this description. Recursion is about solving a problem in terms of smaller versions of itself. The process ends because the problems get smaller until the process reaches some problem small enough to be solved directly.

It would be more accurate, but less humorous, to say that in order to have a good understanding of recursion, you must first have a rough understanding of recursion.

Related post: The secret to understanding recursion