Dialing back the cleverness

Last night I ran into Damian Conway at a speaker’s dinner for this week’s GOTO conference. He’s one of the people I had in mind when I said I enjoy hearing from the Perl community even though I don’t use Perl.

We got to talking about Norris’ number, the amount of code an untrained programmer can write before hitting a brick wall. Damian pointed out that there’s something akin to the Norris’ number for skilled programmers. He said that a few years ago he took this quote from Brian Kernighan to heart:

Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.

He said he’d realized that he’d written some very complex code that was approaching his level of cleverness and that couldn’t move forward himself, much less get anyone else to take it ownership. “That’s when I decided to dial back the cleverness a bit.”

Related post: Two kinds of software challenges

8 thoughts on “Dialing back the cleverness

  1. Cleverness… I’m not actually totally sure what that is.

    I remember, once upon a time, I would try to minimize the size of my code, or make it faster, or that kind of thing. After a certain point, I imagine my code was “clever”? I know for certain that squeezing the last 5% of speed or size or whatever out of my code would almost invariably result in something that was painful to work with when I discovered something that had to change.

    Anyways, the issue here, I imagine, is abstractions — our code represents concepts which (in a complex system) we are discovering as we go along. (If we could start with a perfect specification, that properly addressed all ambiguities, and we could start with a complete understanding of that specification, and if we could guarantee that the specification never changed and we could prevent new requirements from coming in…. we would probably have to stop all technological progress to achieve that.)

    So, anyways, we have discovery process. processes. discovery without process. new stuff…

    So that means we need to build systems that can change.

    Breaking that down:

    1. Before the change, our system is inadequate (broken in some new way but perhaps not broken in the old way).

    2. After the change, our system is different (perhaps not broken in some new way, but if things were perfect before they are not that way now).

    So, from my point of view, this process of adaption is at cross purposes with concepts of perfection. And, ironically, in my experience, the worst possible way to deal with this change process is to “design for the future”. The problem here, is that I never have the requirements that are going to arrive later. And when I try to design for them I invariably pick the wrong design. So, instead, the best thing to do is go for the simplest possible thing that works.

    Simplicity brings with it some other advantages: it’s usually relatively short (since I’m leaving out all the complex stuff), and it’s usually rather fast (since I’m leaving out a bunch of unnecessary stuff).

    Note also that some judicious use of mathematical abstractions can be very helpful when simplifying. (But please do not assume that by “simple” I mean “rote” or “uninformed” or “obvious to someone who does not understand”. Nothing is obvious to someone that does not understand unless they are looking at the wrong thing — obviousness is a concept used to describe stuff we do understand.)

    So, anyways… simplicity is good. And when cleverness takes you someplace that’s not simple? That is not so good.

    But I’m not prepared to say that cleverness is bad. After all, my feeling is that making things simple also requires cleverness. My hunch is that cleverness (whatever that is) is not the problem but that the directions we give our cleverness are the problem.

    Or, since I’ve gotten long-winded here, here’s the short form:

    Keep it Simple.

  2. I think of it as the elevator pitch now. If you can’t give an elevator pitch, a brief 30 to 60 second explanation, for your code at each level of abstraction, than perhaps you took it too far.

  3. Speaking of Perl, Perl 6 turned into a morass of cleverness and second-system effect. It not only includes everything everyone wanted, it does it in torturously clever ways. I think this has a lot to do with the lack of adoption by real software developers.

  4. @rdm I think there’s a particular type of cleverness that this refers to that is hard to describe but not so hard to recognize.

    It’s the code where when someone looks at it they ask “why does this work” (which is already a bad sign) and you answer with “well you have to think about it like ” and the asker starts furrowing his brow and rubbing his eyes. Maybe when this is all done, if you’re lucky, he’ll say “oh I do understand… yeah, that’s quite clever.”

  5. @StevenNoble yes…

    …but sometimes that’s justified. Using quaternions and/or eigenvalues in graphics programming, for example, can sometimes be useful examples of “clever”.

    All too often, though, it’s pointless. That’s when it’s bad.

    And that’s the real problem, I think — when we are doing useless stuff, or doing things for useless reasons.

    But, yes, also: when describing our code is difficult it’s often much easier to change the code to make it easy to describe than it is to describe the unchanged code. And yet, I almost never see “writing documentation for ___ audience” as an architectural or design issue in any of the development projects I’ve been working on.

  6. I wrote some code that generates other code using templates. The core was one line, something like print(subst(read(open(“template.txt”)))). The templates are written in the target language with nested code that gets eval’d by subst(). It was a clever solution but I have a feeling that some other programmer eventually will need to modify it, won’t understand the substitution rules, and will take it out and replace it with something easier to understand, like regexp substitution.

  7. When programmers talk about “cleverness” they usually mean a very specific type of code. Usually what this means is using some obscure language feature in a novel way so that it is not readily apparent what the code is doing. This can often lead to code that looks very simple and “nifty”. The problem is that such code often doesn’t generalize well and since it is not readily apparent what it does, if someone else tries to change or maintain it, it breaks easily. To me, a classic example of this would be the C version of Duff’s device. It’s a nifty way to unroll a loop, but it’s harder to understand and maintain than a regular loop and may even decrease performance.
    For me, there are good and bad kinds of cleverness in code.
    1. More abstract algorithms when solving a problem since these can result in cleaner implementations.
    2. More abstract and decoupled classes / functions.
    1. Cleverness by way of using obscure language features and optimizations when not strictly necessary.
    2. Saving yourself typing at the expense of increased code coupling. For example deep inheritance hierarchies.

  8. Whenever possible, I write very simple, easy-to-understand code, with lots of narration in the form of comments.

    I do this because I know that the person who will try to debug that code is very dense and slow. You have to spoon feed him or he gets all mixed up.

    Since I want that guy to succeed, I try to tell him everything he needs to understand what I am doing.

Comments are closed.