Python / Emacs setup

When I got a new computer a few days ago, I installed the latest version of Emacs, 24.2, and broke my Python environment. I decided to re-evaluate my environment and start over. I asked a question on the Python Google+ group and here’s a summary of the Emacs packages recommended.

  • python-mode
  • rainbow-mode, rainbow-delimiters-mode
  • flymake-mode (hooked up to flymake-pyflakes)
  • linum-on
  • jedi
  • pycheckers + pyflakes
  • rope
  • electric-pair
  • show-paren
  • auto-complete
  • yassnippets

What recommendations do you have for packages? Links to relevant articles?

It might be better to leave your comments on Google+ rather than here so the answers will be in one place.

It’s not the text editor, it’s text

Vivek Haldar had a nice rant about editors a couple days ago. In response to complaints that some editors are ugly, he writes:

The primary factor in looking good should be the choice of a good font at a comfortable size, and a syntax coloring theme that you like. And that is not something specific to an editor. Editors like Emacs and vi have almost no UI!

To illustrate his point, here’s what my Emacs looks like without text:

There’s just not much there, not enough to say it’s pretty or ugly.

When people say that Emacs is not pretty, I think they mean that plain text is not pretty.

For better and for worse, everything in Emacs is text. The advantage of this approach is consistency. Everything uses the same commands for navigation and editing: source code, error messages, directory listings, … Everything is just text. The disadvantage is that you don’t have nicely designed special windows for each of these things, and it does get a little monotonous.

When people say they love their text editor, I think they love text. They prefer an environment that allows them to solve problems by editing text files rather than clicking buttons. And as Vivek says, that is not something specific to a particular editor.

Related posts:

Editing by semantic units

The most basic text editor commands operate on lines and characters: move up or down a line, delete the next or previous character, etc. More advanced commands operate on context-specific semantic units. In the context of English prose, this means words, sentences, and paragraphs. In the context of source code, it means blocks and functions, however these are delimited in a particular language. In the context of an outline, it means navigating the tree.

One way to become more proficient at using your editor of choice is to become fluent at working with larger semantic units. Of course this saves keystrokes, though as I argued here, I don’t think typing speed matters so much. However, working in larger semantic units reduces errors. It also improves the chances that you can act on an idea before you forget what you were doing. These factors matter more than typing speed per se.

The general principles above apply to any editor. Here I’ll sketch how these ideas work in Emacs.

As a general rule, commands that operate on physical units begin with Control and analogous commands that work on semantic units being with Meta (Alt). For example, C-f moves forward a character and M-f moves forward a word. C-e moves to the end of a line and M-e moves to the end of a sentence.

Emacs extends the meaning of prose units by analogy in other contexts. For example, in calendar mode, a “line” corresponds to a week, and a “paragraph” corresponds to a month.

Emacs has numerous commands to work on “balanced expressions,” i.e. text surrounded by parentheses, brackets, braces, quotation marks, etc. Often the default keybindings are consistent with those used for other units. For example, “f” means forward and “b” means backward in any context. C-f and C-b move by character, M-f and M-b move by word, and C-M-f and C-M-b move by balanced expression. Emacs also has commands for working with function definitions: selecting a function definition, moving to the beginning or end of a function definition, etc. In Lisp, function definitions are balanced expressions. In other languages, like C, these ideas are distinct.

This morning, Irreal points out an Emacs package expand-region that selects larger and larger semantic regions. The first time it is invoked, it selects the immediate context of the cursor. Each subsequent invocation selects a larger context, moving up the parse tree of the content. I look forwarding to trying it.

* * *

For daily tips on using Unix, follow @UnixToolTip on Twitter.

UnixToolTip twitter icon

How Emacs influenced Ruby

Ruby creator Yukihiro Matsumoto gave a presentation How Emacs changed my Life in which he explains how Emacs influenced him personally and how it influenced the programming language he created. Here is his summary:

  1. Emacs taught me freedom for software.
  2. Emacs taught me how to code.
  3. Emacs taught me the power of Lisp.
  4. Emacs taught me how to implement a core language.
  5. Emacs taught me how to implement a garbage collector.
  6. Emacs helped me to code and debug.
  7. Emacs helped me to write and edit text/mails/documents.
  8. Emacs helped me to be an effective programmer.
  9. Emacs made me a hacker.
  10. Emacs has changed my life.

Shuffling Emacs windows

Emacs lets you split your screen into windows, what other applications might call panels. This can be quite handy. However, I often want to move the windows around and I couldn’t find how to do that. I asked Xah Lee, he posted the question on G+, and Mark Hepburn answered with a pointer to code that does what I wanted: buffer-move.el

(Strictly speaking the windows don’t move; the buffers move between the windows. But I think of it as moving the windows around.)

Lucas Bonnet explains how his buffer-move works:

This file is for lazy people wanting to swap buffers without
typing C-x b on each window. This is useful when you have :

|              |             |
|    #emacs    |    #gnus    |
|              |             |
|                            |
|           .emacs           |
|                            |

and you want to have :

|              |             |
|    #gnus     |   .emacs    |
|              |             |
|                            |
|           #emacs           |
|                            |

This is exactly what I had in mind. I did have one problem, however, though it wasn’t with buffer-move per se. The file defines four functions for moving buffers between windows and suggests key bindings. The author says he uses the following.

(global-set-key (kbd "<C-S-up>")     'buf-move-up)
(global-set-key (kbd "<C-S-down>")   'buf-move-down)
(global-set-key (kbd "<C-S-left>")   'buf-move-left)
(global-set-key (kbd "<C-S-right>")  'buf-move-right)

This works fine, except when one of the windows contains an org-mode buffer. Org-mode has its own bindings for control-shift-up etc.

I learned that global-set-key does not globally set a key binding! Well, it does as long as there are no conflicts. But local bindings, i.e. bindings specific to a mode, have precedent. After some searching and trail-and-error, I added the following to my .emacs to override org-mode’s local bindings.

(add-hook 'org-mode-hook '(lambda ()
   (local-set-key [C-S-up]    'buf-move-up)
   (local-set-key [C-S-down]  'buf-move-down)
   (local-set-key [C-S-left]  'buf-move-left)
   (local-set-key [C-S-right] 'buf-move-right)))

Clipboard history

The Windows clipboard only remembers the most recent thing you copied [1]. This can be very disappointing. Maybe you cut a large block of text intending to paste it somewhere, but without thinking you cut something else, and then realize you’ve just wiped out your first cut. Here are a few possible alternatives for Windows, MS Office, and Emacs.

A few years ago I tried ClipX, a free clipboard history manager for Windows, based on Jeff Atwood’s recommendation. It worked OK at first, but then I had problems with it and stopped using it. It looks like the software hasn’t been updated since 2008. If you know of a better alternative, please let me know. Also, please let me know if you have suggestions for Ubuntu.

(Update: See the comments for numerous suggestions of ClipX-like products. Also, see PHenry’s comment about clipboard history in Visual Studio.)

MS Office has its own clipboard that will store 24 entries. It stores things that have been copied either inside or outside of Office applications, but you can only paste from the Office clipboard using an Office application.

Emacs has a “kill ring” which is essentially a clipboard. It also has “registers” which are like named clipboard items.

Yesterday Emacs Knight posted a handy little bit of code for viewing your kill ring.

    (global-set-key "C-cy" '(lambda ()
        (interactive) (popup-menu 'yank-menu)))

If you add this to your .emacs file, you can type ctrl-c y to pull up a little GUI menu displaying the contents of your kill ring.

[1] I’ve heard that Windows actually keeps more clipboard history, but it doesn’t provide a user interface for accessing that history. I don’t know whether this is true. But if it is, you could access the history programmatically. Maybe Office isn’t maintaining its own clipboard but just exposing more of the system clipboard.

Playing with Eshell

I’ve been trying out Eshell lately. It’s a shell implemented in Emacs Lisp. Here I’ll mention a few features I’ve found interesting.

The command M-x shell lets you run a shell inside Emacs. This is quite handy, but it runs a different shell on each platform. For example, it might pull up bash on Linux and cmd.exe on Windows. Eshell provides a consistent shell across operating systems.

Eshell is one way to have a bash-like shell on Windows, so it’s an alternative to Cygwin. I haven’t used Eshell or Cygwin extensively, but Cygwin feels like some alien thing awkwardly grafted on to my Windows machine. Eshell feels more like cmd.exe with bash features added.

Of course Eshell is integrated with Emacs. One example of that is being able to run Emacs commands as shell commands. For example, you can run dired from the shell to open a directory in the Emacs directory editor.

Eshell has pseudo-devices. These act like Unix devices, but they’re implemented in Emacs and therefore run on non-Unix platforms. For example, you could direct command output to /dev/clip to send it to the system clipboard or to  /dev/kill to add it to the Emacs kill ring.

And now for a little silliness. Here’s an excerpt from Tron featuring Eshell.

It goes by very fast, but here’s the frame that shows Eshell:

eshell screen shot

For much more on Eshell, see A Complete Guide to Mastering Eshell.

Mixing R, Python, and Perl in 14 lines of code

This is a continuation of my previous post, Running Python and R inside Emacs. That post shows how to execute independent code blocks in Emacs org-mode. This post illustrates calling one code block from another, each written in a different language.

The example below computes sin2(x) + cos2(x) by computing the sine function in R, the cosine function in Python, and summing their squares in Perl. As you’d hope, it returns 1. (Actually, it returns 0.99999999999985 on my machine.)

To execute the code, go to the #+call line and type C-c C-c.

#+name: sin_r(x=1)
#+begin_src R

#+name: cos_p(x=0)
#+begin_src python
import math
return math.cos(x)

#+name: sum_sq(a = 0, b = 0)
#+begin_src perl
$a*$a + $b*$b;

#+call: sum_sq(sin_r(1), cos_p(1))

Apparently each function argument has to have a default value. If that’s documented, I missed it. I gave the sine and cosine functions default values that would cause the call to sum_sq to return more than 1 if the defaults were used.

Running Python and R inside Emacs

Emacs org-mode lets you manage blocks of source code inside a text file. You can execute these blocks and have the output display in your text file. Or you could export the file, say to HTML or PDF, and show the code and/or the results of executing the code.

Here I’ll show some of the most basic possibilities. For much more information, see And for the use of org-mode in research, see A Multi-Language Computing Environment for Literate Programming and Reproducible Research.

Source code blocks go between lines of the form


On the #+begin_src line, specify the programming language. Here I’ll demonstrate Python and R, but org-mode currently supports C++, Java, Perl, etc. for a total of 35 languages.

Suppose we want to compute √42 using R.

#+begin_src R

If we put the cursor somewhere in the code block and type C-c C-c, org-mode will add these lines:

: 6.48074069840786

Now suppose we do the same with Python:

#+begin_src python
from math import sqrt

This time we get disappointing results:

: None

What happened? The org-mode manual explains:

… code should be written as if it were the body of such a function. In particular, note that Python does not automatically return a value from a function unless a return statement is present, and so a ‘return’ statement will usually be required in Python.

If we change sqrt(42) to return sqrt(42) then we get the same result that we got when using R.

By default, evaluating a block of code returns a single result. If you want to see the output as if you were interactively using Python from the REPL, you can add :results output :session following the language name.

#+begin_src python :results output :session
print "There are %d hours in a week." % (7*24)

This produces the lines

: There are 168 hours in a week.
: 1024

Without the :session tag, the second line would not appear because there was no print statement.

I had to do a couple things before I could get the examples above to work. First, I had to upgrade org-mode. The version of org-mode that shipped with Emacs 23.3 was quite out of date. Second, the only language you can run by default is Emacs Lisp. You have to turn on support for other languages in your .emacs file. Here’s the code to turn on support for Python and R.

    'org-babel-load-languages '((python . t) (R . t)))

Update: My next post shows how to call code in written in one language from code written in another language.

Related posts:

Advantages of everything-is-text

In a typical Windows program, some output text can be copied as text but some is only an image. For example, the text in the body of a document is text, but the words on the menus are not. The text in a dialog box may or may not be.

In Emacs, text is just text, no matter where you see it. This means that Emacs is less visually interesting because there is less variety. But this also has several advantages that may not be immediately obvious. I’ll give a couple examples.

The Emacs directory editor dired lists files in a text buffer. The output is essentially the output of ls. (On Windows, Emacs emulates ls.) That means you can do anything to the output you could do to text: copy, paste, search, etc. No text is off-limits. And in editable mode, you can rename files just by editing their names. You could, for example, rename many files at once using regular expressions.

Dired also lets you navigate the file system with the same commands you use to edit files. In particular, you can keep your hands on the keyboard. (It’s possible to use the Windows file explorer without a mouse, but it’s limited and awkward.)

Another advantage of everything simply being text is that you can run a shell inside Emacs. Why not? A shell is just text output. In an ordinary shell, editing commands are mostly limited to the current line. Not all text is created equal. But when you’re running a shell inside an editor, all text is equal: current line, previous lines, and text on either side of the command prompt. And if you want a transcript of a shell session, just save the buffer displaying the session.

If you’re shopping for a text editor and spend an hour trying each one out, Emacs may not look very promising. It has odd terminology, weird key strokes, etc. But if you try Emacs long enough, and particularly if you have someone to help you get started, you start to appreciate the advantages of subtle features such as everything simply being a text buffer.

Related posts: