Computing

Google’s Urs Hölzle gives the following world-wide average statistics regarding internet use.

• Average page load: 4.9 seconds
• Average page size: 320 kilobytes
• Average bandwidth: 225 kilobytes/second

If bandwidth were the only limitation, the average page should load four times faster using the average bandwidth. The internet protocols that have served us remarkably well were designed for very different usage scenarios. Hölzle says that web pages could load between two and four times faster if we make slight changes to infrastructure protocols and their implementations.

(Sam Savage would point out that you can get into trouble using averages as we did above. When you have variable quantities X and Y, the average of X/Y is not simply the average of X divided by the average of Y. But the calculations above are accurate enough for back-of-the-envelope estimates.)

A while back I asked Jason Fruit how to add HTML tags to text in Emacs, something like the format painter in Microsoft applications. He said he didn’t know of anything and wrote the following macro for me.

(defun tag-word-or-region (tag)
    "Surround current word or region with a given tag."
    (interactive "sEnter tag (without <>): ")
    (let (pos1 pos2 bds start-tag end-tag)
        (setq start-tag (concat "<" tag ">"))
        (setq end-tag (concat "</" tag ">"))
        (if (and transient-mark-mode mark-active)
            (progn
                (goto-char (region-end))
                (insert end-tag)
                (goto-char (region-beginning))
                (insert start-tag))
            (progn
                (setq bds (bounds-of-thing-at-point 'symbol))
                (goto-char (cdr bds))
                (insert end-tag)
                 (goto-char (car bds))
                 (insert start-tag)))))

I added the following line to my .emacs file to bind Jason’s macro to the key sequence C-x t.

(global-set-key "\C-xt" 'tag-word-or-region)

To add a tag to a single word, place the cursor before or in the word and execute the command. To tag a block of text, select the text first then execute the command.

Related posts:

Emacs adventures
Giving Emacs another try

I recently found out there’s an Emacs command M-x woman that’s a pun on “w/o man”, i.e. a way to read online help without using the usual man command.

***

I tried to edit a 1.2 GB text file with Emacs the other day. I got an error saying that Emacs has a 500 MB file size limit on 32-bit systems. This was on a 32-bit Windows XP machine, so the warning was reasonable. I appreciated that it promptly said it could not open the file.

I tried again on my 64-bit Windows 7 machine and got the same message; I believe I’m running a 32-bit version of Emacs on my 64-bit PC.

I also tried opening the file with Emacs on a 64-bit Red Hat Enterprise Linux box with 16 GB of memory. This time I did not get a file size message. Instead, Emacs crashed.

The file opened quickly on Windows, even my 32-bit XP box, using Notepad++.

***

When I set up Emacs on Windows, I created an “Open with Emacs” context menu following the instructions here. That has worked well except for two problems.

1. Every file is opened in a new instance of Emacs.
2. It stopped working on one of my computers for reasons unknown.

I’ve been meaning to solve the first problem, and when the second happened I decided it was time. According to the Emacs documentation,

The recommended way to associate files is to associate them with
emacsclientw.exe. In order for this to work when Emacs is not yet started, you will also need to set the environment variable ALTERNATE_EDITOR to runemacs.exe. To open files in a running instance of Emacs, you will need to add the following to your init file: (server-start)

I had to tinker with that a little to make it work. I know I had to add the path to my Emacs bin directory to my PATH environment variable but I don’t remember what else I did.

The key thing to note is that you should associate files with emacsclientw.exe rather than with runemacs.exe. I used the instructions in this article to remove the association with the latter. Thanks to Mark for pointing out the article.

(The Windows “Open with” feature remains a mystery. The article above explains how to remove a program from the “Open with” list, and the directions worked for removing runemacs.exe. But they did not work for removing other programs. I’m also unclear how you add something to the “Open with” list. If you right-click on a file and select “Open with -> Choose program …” the program you select may appear in the “Open with” list next time. Or not. This did add emacsclientw.exe to the list.)

Related post:

Emacs a few weeks later
Not for everyone

I remap the Caps Lock key to be a control key on every computer that I use regularly. Here’s why.

1. Caps Lock is a nearly worthless key taking up valuable real estate.
2. I’m more likely to use Caps Lock accidentally than intentionally.
3. I use the Control key far more often than the Caps Lock key.
4. The Caps Lock key is in a consistent position on all keyboards but the left Control key isn’t.

Some people swap the Caps Lock and and left Control keys. I prefer to just disable the left Control key. On desktop keyboards, I map the useless Scroll Lock key to act as a Caps Lock key for those rare instances when I actually want to type a long sequence of capital letters.

KeyTweak is a convenient program for remapping a Windows keyboard.

On Linux, you can use the xmodmap utility.The following commands disable the left Control key and turn the Caps Lock key into a control key.

xmodmap -e "remove Lock = Caps_Lock"
xmodmap -e "remove Control = Control_L"
xmodmap -e "keysym Caps_Lock = Control_L"
xmodmap -e "add Control = Control_L"

See Dave Richeson’s comment below for Mac instructions.

Related post:

41 dumb things to check

A decade ago, commercial software vendors would claim that their products were cheaper than open source alternatives when you considered the total cost of ownership. Free software was free to obtain, but difficult to install, configure, maintain, and support.

A lot has changed in the last decade. Open source software has improved a great deal. It would be interesting to revisit the debate over total cost of ownership. Software vendors are right to point out the indirect costs of free software. But there are indirect costs to commercial software too: transaction costs of purchasing the software, upgrades, maintenance agreements, license management, etc.

Suppose you want to buy WinZip. It’s a mature and inexpensive piece of software, selling for $29.95. What will it cost you and your company to buy it? Obviously at least $29.95. But how much paperwork will you have to fill out? How long will it take someone to process your order? How long will you have to wait? If you have a desktop and laptop computer, will you be licensed for both? Can you install it at home? At minimum you’ll have to read enough fine print to find out. Now suppose you get a new PC. Did you remember to save your WinZip installer before they took away your old PC? Do you have your license key? The more you think about it, the better the free alternative 7-zip looks.

Related posts:

Shallow bugs versus reported bugs
Rules for computing happiness
“Noncommercial” is fuzzy

Unix philosophy says a program should do only one thing and do it well. Solve problems by sewing together a sequence of small, specialized programs. Doug McIlroy summarized the Unix philosophy as follows.

This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.

This design philosophy is closely related to “orthogonality.” Programs should have independent features just as perpendicular (orthogonal) lines run in independent directions.

In practice, programs gain overlapping features over time.  A set of programs may start out orthogonal but lose their uniqueness as they evolve. I used to think that the departure from orthogonality was due to a loss of vision or a loss of discipline, but now I have a more charitable explanation.

The hard part isn’t writing little programs that do one thing well. The hard part is combining little programs to solve bigger problems.  In McIlroy’s summary, the hard part is his second sentence: Write programs to work together.

Piping the output of a simple shell command to another shell command is easy. But as tasks become more complex, more and more work goes into preparing the output of one program to be the input of the next program. Users want to be able to do more in each program to avoid having to switch to another program to get their work done.

For example, think of the Microsoft Office suite. There’s a great deal of redundancy between the programs. At a high level, Word is for word processing, Excel is the spreadsheet, Access is the database etc. But Excel has database features, Word has spreadsheet features, etc. You could argue that this is a terrible mess and that the suite of programs should be more orthogonal. But someone who spends most of her day in Excel doesn’t want to switch over to Access to do a query or open Word to format text. Office users are grateful for the redundancy.

Software applications do things they’re not good at for the same reason companies do things they’re not good at: to avoid transaction costs. Companies often pay employees more than they would have to pay contractors for the same work. Why? Because the total cost includes more than the money paid to contractors. It also includes the cost of evaluating vendors, writing contracts, etc. Having employees reduces transaction costs.

When you have to switch software applications, that’s a transaction cost. It may be less effort to stay inside one application and use it for something it does poorly than to switch to another tool. It may be less effort to learn how to do something awkwardly in a familiar application than to learn how to do it well in an unfamiliar application.

Companies expand or contract until they reach an equilibrium between bureaucracy costs and transaction costs. Technology can cause the equilibrium point to change over time. Decades ago it was efficient for organizations to become very large. Now transaction costs have lowered and organizations outsource more work.

Software applications may follow the pattern of corporations. The desire to get more work done in a single application leads to bloated applications, just as the desire to avoid transaction costs leads to bloated bureaucracies. But bloated bureaucracies face competition from lean start-ups and eventually shed some of their bloat or die. Bloated software may face similar competition from leaner applications. There are some signs that consumers are starting to appreciate software and devices that do less and do it well.

Related posts:

Organization scar tissue
Visualizing software development effort
Moore’s law and software bloat

The expression “_____ isn’t for everyone” can sound snobbish. For example, if someone says that their favorite wine isn’t for everyone, are they really saying that not everyone has the refined taste that they do? Or if they say their favorite author isn’t for everyone, are they really saying that not everyone is smart enough to appreciate the books they enjoy?

But sometimes the expression is used humbly, as in Brian Carper’s article Emacs isn’t for everyone. There are some arrogant Emacs users out there, but Brian isn’t one of them, at least not as far as I can tell by reading his article. He simply discusses some of the pros and cons of the software and explains why some people would be better served by another tool.

I hope you’ll keep reading even if you have no interest in Emacs. This isn’t a post about Emacs per se. It’s about some of the things the article made me think of.

The majority of software development articles either advocate the author’s tool of choice or complain about a tool the author is forced to use. (See Ford-Chevy arguments in tech for a possible explanation.) Articles that frankly discuss pros and cons are rare. Here’s a sentence from the article I particularly liked.

Mastering an arcane text editor isn’t necessarily going to be on the top of the list of everyone’s goals in life, especially when there are other editors that are easier to use and give you a significant subset of what Emacs would give you.

That’s a far cry from “Maybe you’re not smart enough to use Emacs.”

I also appreciated that the article wasn’t overly pragmatic. It wasn’t just a dry cost-benefit analysis. Brian explains

I didn’t learn Emacs with the goal of being productive. I learned it for the same reason some people build cars in their garages, while most people just buy a one and drive it to and from work every day. … For me, productivity was a beneficial side-effect.

Every once in a while you’ll see someone say they use a tool for fun. For instance, I’ve heard several people say they were burned out on programming until they discovered Perl or Ruby. But it’s far more common for someone to say “This made me more productive.” than to honestly admit “I enjoy tinkering with this.” The two probably go hand in hand: you’ll probably be more productive using a tool you enjoy tinkering with even if in some objective sense it’s inferior to another tool.

Related posts:

Doing good work with bad tools
Chauffeurs and Ferraris revisited
Emacs a few weeks later

The shelf life of software development books is typically two or three years, maybe five or ten years for a “classic.” Frederick Brooks, however, wrote a book on software development in 1975 that remains a best-seller: The Mythical Man-Month. His book has remained popular because he wrote about human nature as applied to software development, not the hottest APIs and development fads from the 1970’s.

Frederick Brooks has written a new book that should also enjoy exceptional shelf life, The Design of Design: Essays from a Computer Scientist. In this book, Brooks looks back over a long successful career in computing and makes insightful observations about design.

The following interview comes from an email exchange with Frederick Brooks.

JC: You did a PhD in math with Howard Aiken in 1956. Did you study numerical analysis? Did you intend at the time to have a career in computing?

FB: Oh, yes indeed. That’s why I went to Aiken’s lab for graduate work.

JC: I was struck by your comments on conceptual integrity, particularly this quote: “Most great works have been made by one mind. The exceptions have been made by two minds.” Could you say more about that?

FB: I give lots of examples in the book of the one mind case, and a few of the two-mind case.

JC: You said in your book that your best management decision was sending E. J. Codd to get his PhD. That certainly paid off well. Could you share similar examples of successful long-term investments?

FB: Well, IBM’s decision to gamble everything on the System/360, terminating six successful product lines to do so, is a great example. DARPA’s funding of the development of the ARPAnet, ancestor to the Internet, is another great example.

JC: What are some technologies from decades ago that are being rediscovered?

FB: I find it useful to write first drafts of serious things, such as scientific papers and books, by hand with a felt-tip pen. I can type faster than I can think, so composing on a keyboard yields unnecessary wordiness. Writing by hand matches my thinking and writing speeds, and the result is leaner and cleaner.

JC: What are some that have not become popular again but that you think should be reconsidered?

FB: Probably the previous example answers this question better than it does the previous question.

JC: Apart from technological changes, how have you seen the workplace change over your career?

FB: I’ve been in academia for the past 46 years. The biggest change in academia is a consequence of personal computers and networks: faculty members don’t use secretaries as such, they write their own letters, and make their own phone calls. Our assistants are indeed administrative assistants, rather than secretaries.

JC: What change would you like to see happen as a result of people reading your new book?

FB: Even more recognition of the role of a chief designer, separate from a project manager, in making a new artifact, and more attention to choosing and growing such.

Related links:

The Design of Design
The Mythical Man-Month
Many hands make more work

Other interviews:

Cliff Pickover
Carl Franklin
Dan Bricklin

A few weeks ago I mentioned that I would give Emacs another try.  I said I would use it through April and then decide whether to keep using it or give up. Here are some thoughts on Emacs a few weeks later.

[click to continue...]

In his book The Design of Design, Frederick Brooks describes his most productive decision as a manager at IBM.

My most productive single act as an IBM manager had nothing to do with product development. It was sending a promising engineer to go as a full-time IBM employee in mid-career to the University of Michigan to get a PhD. This action … had a payoff for IBM beyond my wildest dreams.

That engineer was E. F. Codd, father of relational databases.

Related post:

Many hands make more work

Ben Jaffe has started a Twitter account @commandtab with regular keyboard shortcuts for the Mac, something similar to my account @SansMouse for Windows. You can hear Ben talk about his new Twitter account on the GeekSpeak radio program and podcast.

Related links:

Four patterns in Windows keyboard shortcuts
Using Windows without a mouse
Twitter daily tip accounts

I used Emacs for several years until 1995 when I started developing Windows software. I tried picking Emacs back up a few times since then, but each time I got frustrated and decided it wasn’t worthwhile. But things change and now I’m willing to give it another try. I’m well aware of the difficulties, but I think it may be worth the investment.

I’ve evaluated several editors on Windows, and I haven’t found anything as powerful or as configurable as Emacs. But the main reason I’m willing to try Emacs again is that I’m tired of using a different program for every kind of file I edit. I use a dozen programs, none of which I know very well. I want to learn one tool well and use it for many different tasks. As the Emacs guided tour says:

All of the basic editing commands (and there are lots of them) are available no matter what you’re trying to do: write code, read a manual, use a shell, or compose an email.

All the tools Emacs provides for opening, saving, searching, and processing text (and more) are available to you no matter what you’re doing.

Using Emacs is painful at this point, like the first day getting back to an exercise routine. But some old habits are starting to coming back, and I’ve discovered a few customizations that make Emacs more convenient.

I’m doing an experiment. I’ve committed to using Emacs through the end of April. After that I may decide that the quirks of Emacs are either too much to get used to or will take too much effort customize around. Or I may decide I want to keep using it and dive deeper.

Related posts:

Emacs
One program to rule them all
Ford-Chevy arguments in tech
Using Windows without a mouse

Emacs is a text editor with ambitions to be an operating system. I do not use Emacs, though I once did, and I still find it intriguing. I’d like to find something similar that acts more like a Windows program.

GNU Emacs began in 1984 and has been in constant development ever since. The current version is 23.1. How many applications from 1984 are still in widespread use today? The only other one that comes to mind is TeX.

I used Emacs in graduate school and for a few years after that. I was fairly fluent with Emacs, though I never customized it much. I intended to learn Emacs Lisp and all that, but it never happened.

When I started developing Windows software I used Emacs at first, but the benefits of Visual Studio soon persuaded me give up my old editor. It was much easier to go with the flow.

I’ve revisited Emacs a couple times over the years. I still have some of the keystrokes burned into my memory. I use it on Linux now and then, but I mostly work on Windows, and my experience using Emacs on Windows has been frustrating to say the least. Tasks that are trivial in any Windows application, such as printing and spell checking, are surprisingly difficult to set up in Emacs. I’m sure it is possible to resolve these problems, though I never did.

The problems with printing and spell checking are part of the larger issue that Emacs is so idiosyncratic. It behaves nothing like a typical Windows program. Some people may say that’s a good thing. But it makes life more complicated if you switch between Emacs and more conventional Windows software.

Emacs is no more a typical Mac application than it is a typical Windows application. And yet my impression is that this is less of a problem for Mac users. I’d like to understand whether this is true and if so why.

One of the things I liked about Emacs was the way you could “live” there. An expert Emacs user might work inside Emacs all day, using it as an editor, debugger, shell, file system explorer, email program, etc. Steve Yegge is such an expert. When he blogged about his move from Windows to Mac,  he said the main reason for the switch was that he prefers the appearance of the fonts on a Mac. Changing operating systems was not a big deal for Yegge because he didn’t really live in Windows before, nor does he live in OS X now. He lives in Emacs. He concluded his essay by saying

So I’ll keep using my Macs. They’re all just plumbing for Emacs, anyway. And now my plumbing has nicer fonts.

Living inside Emacs comes at a price. Part of that price is writing lots of Emacs Lisp to glue things together. Another part of that price is the commitment to practicing using Emacs. As Yegge says elsewhere

… you need to make a serious, lifelong commitment to Emacs in order to master it. … So it’s not an editor for the faint of heart …

Yikes! I’m not ready to make a serious, lifelong commitment to a piece of software. To my wife? Yes. To my text editor? No.

One of the best features of Emacs is that it has custom “modes” for various kinds of files. Instead of using a separate program for editing every kind of file, Emacs users use one program with different modes. As soon as a new file type comes out, say for a new programming language, someone will post an Emacs mode for that new language.

I’d like to find an editor on Windows that is analogous to Emacs. By that I mostly have in mind a powerful, highly configurable editor with support for many file types. I’d want it to behave like a Windows application, not a foreign transplant, and integrate well with .NET.

There was a project to create such an editor, nicknamed Emacs.NET. It was announced in late 2007. It sounds like the project is still alive, but it doesn’t seem all that promising.

I’ve looked at a few Windows editors that claim to be highly configurable but are not well documented. So if such an editor is configurable, it’s configurable for the person who wrote it or possibly for anyone else willing to study the source code.

Any suggestions for a general purpose Windows editor? For starters, I’d be pleased to find something that’s good at editing LaTeX and HTML.

Update (2 April 2010): I’ve decided to give Emacs another try.

Related posts:

This post started out as an update to my earlier post One program to rule them all.

Virtual reality pioneer Jaron Lanier writes in his book You Are Not a Gadget about the lack of creativity in our use of computing power.

Let’s suppose that back in the 1980s I had said, “In a quarter century, when the digital revolution has made great progress and computer chips are millions of times faster than they are now, humanity will finally win the prize of being able to write a new encyclopedia and a new version of UNIX!” It would have sounded utterly pathetic.

The quote specifically alludes to Wikipedia and Linux, but Lanier is critical of web culture in general. I’m not sure what I think about his position, but at a minimum he provides a counterbalance to the people who speak about the web in messianic tones.