Is helpful software really helpful?

In his new book The Shallows, Nicholas Carr relates an experiment by Christof van Nimwegen on computer-human interaction. Users were asked to solve a puzzle using software. Some users were given software designed to be very helpful, highlighting permissible moves etc. Other users were given bare-bones software.

In the early stages of solving the puzzle, the group using the helpful software made correct moves more quickly than the other group, as would be expected. But as the test proceeded, the proficiency of the group using the bare-bones software increased more rapidly. In the end, those using the unhelpful program were able to solve the puzzle more quickly and with fewer wrong moves.

I immediately thought of the debate over fancy software development tools versus simple tools. Then I read the conclusion:

… those using the unhelpful software were better able to plan ahead and plot strategy, while those using the helpful software tending to rely on simple trial and error. Often, in fact, those with the helpful software were found “to aimlessly click around” as they tried to crack the puzzle.

That really sounds like software development.

Christof van Nimwegen did variations on his experiment and got similar results. For example, he had two groups schedule a complicated series of meetings. One group had plain calendar software and one had software designed to help people schedule complicated meetings. The folks with the simple software won.

The debate over whether to use fancy software development tools (e.g. integrated development environments, wizards, etc.) or simple tools (editors and make files) is a Ford-Chevy argument that won’t go away. I could imagine many valid objections to the applicability of the van Nimwegen studies to the software tools debate, but I’d say they score a point for the simple tools side.

A rebuttal to the van Nimwegan studies is that he has only shown that particular helpful software wasn’t particularly helpful. Maybe the specific puzzle-solving software didn’t help in the long run, but someone could have written software that was ultimately more helpful than the bare-bones software. Maybe someone could have written scheduling software that allows people to schedule tasks faster than using simple calendar software.

A rebuttal to the rebuttal is that someone might indeed write software that allows users get the job done more quickly than they would using simpler software. It may even be inevitable that someone will write such software eventually. However, most attempts fail. It’s hard to write genuinely helpful software. Attempts to help a user too much may interfere with the user’s ability to form a good mental model of the problem.

Related post:

Would you rather have a chauffeur or a Ferrari?

Tagged with: ,
Posted in Software development
15 comments on “Is helpful software really helpful?
  1. Mike Swaim says:

    I think that the biggest argument in favor of “helpful” software has been autocompletion with modern frameworks. I often know roughly what I’m looking for, but don’t know the exact function signature that I’m looking for (or even what variants are available).
    And the complaint against software tools is typically against “wizards” that spit out code where you have no idea what it’s generated or how it works. The old Windows Developers Journal (I think) reccomended >1< MFC book because it was the only one they could find that didn't use code generation wizards.

  2. Joe McTee says:

    Makefiles are simple? I would argue that this is more familiarity than simplicity. I suspect a developer who has never seen a makefile before might find them daunting. The same for “editors”. While notepad is simple, it is not a good programmer’s editor. emacs and vi, on the other hand, are good programmer’s editors (for many, neither is my cup of tea), but hardly fit the bill for simple.

    The puzzle example given in your blog represents only one type of problem faced by a developer and maybe not the most common. A full blown IDE such as Eclipse could be considered a simple tool for other types of problem. One that comes to mind is refactoring legacy code.

    Not really arguing with the basic premise of the blog, just some of the specifics.

  3. thomas says:

    hm. i do wish to have helpful software for research. wouldn’t it be great if you had a piece of software that scans all your papers and downloads references automatically, scans those and does an online search, summarizes the results, links the papers, shows the connections between the areas and is integrated in your IDE or editor? when you program in… java a concurrency problem it analyses your type of problem and suggests websites, books, papers etc on the fly.

    i wouldn’t mind that to be honest.

  4. Nick says:

    Really I think this is a issue with technology period. My wife can barely drive to the mall because she has become so dependent on her GPS. It really depends on the person. That is why I think it is best for people to start with the bare bones then use the more advanced. Math by pencil and paper before math with calculator.

  5. Amber Shah says:

    You touched on this in the post, but I wonder if this isn’t just a warning against unhelpful software disguised as helpful software, rather than bashing all good UI.

    And then you also have the fact that those users who took longer to get up to speed may have just abandoned the game before getting proficient. So maybe the answer is to show “helpful” things for new users but not for all users.

  6. JD Long says:

    This morning I’ve been thinking about all the abuses of Excel I see. I’ve started referring to these collectively as “Excel Exceedance Syndrome” or EES. EES generally follows the pattern of folks using Excel because they are familiar with it when another tool (like a programming language or database) would be superior in all ways except for 1; getting started. That’s what so many GUIs really are doing; they kick start the getting started bit. Then often they get in the way of many other bits. Often a GUI helps the user feel like the the task at hand is more well defined and less abstract. Even if this is an artificial security…

  7. The holy grail for UI designers is to have an app that supports the transition from novice to expert. Early on you want lots of affordances; later you want lots of shortcuts. Different users have different preferences and different learning styles and rates.

    Not easy to do well, but a well known problem.

  8. Manoel Galdino says:

    It seems to me that this is related to local maximum. When people have aims and some contraints, they will move to the local maximum (at least, they will try).
    Some tools may help you to find a local maximum faster, but they will trap you there, because in order to go to a global maximum (or a higher local maximum) people will have to decrease their performance.

    That’s what I think that explains the result of the experiment. The “helpful software” trapped people in local maximum. And people with the bare-bones software had to master some skills in order to do the task well. This prompted them to a global maximum (or another local maximum, higher than the other).

  9. Gene says:

    I’ve long been a proponent of learning your base software development tool and using it properly, no matter what the base tool happens to be. I get a kick out of the various frameworks, wizards and code generators for VS 2008. What I’ve determined is that the vast majority of these “helper” tools make getting started a lot easier, but make transitioning to real applications a lot harder.

    There can be no substitute for reading documentation and learning how to develop an application from scratch. Looking up cookbook code on Google saves a lot of time, but does it make us better developers?

  10. Mike Swaim says:

    But Excel is a development environment. It’s a really odd one, using cells, rather than lines of code, but it’s definately a development environment.
    (I once worked on an n-tier system with Excel as one of the front ends. We had spreadsheets that users used to load data into our system. We also had spreadsheets that we pushed data into in near real time. The users loved it.)

  11. Kevin Webber says:

    … and there’s a whole other category of software that’s both unhelpful AND packed with features. I’m thinking of starting a self-help group for developers who have been exposed to ClearCase for too long.

  12. Damian says:

    Excellent article! I actually think that the same argument could be made against computers, the internet, reference manuals, calculators and even GPS when the using becomes a go to answer crutch rather than a simple learning tool. Sadly, it is a propensity of humanity to simple fall back to relying on such things (myself included) rather than putting forth any effort to actually learn. We become so reliant on them that when they are taken away we become helpless.

  13. Michael Fitzmaurice says:

    I think if your IDE is doing things for you that you don’t know how to do without the IDE, that is not ideal. But there is nothing that, say, Eclipse, does for me when I am writing Java code that I couldn’t do for myself from the command line or using some other, more primitive tool. However, the IDE saves me time and effort and actually makes more complex tasks easier for me, rather than harder.

    So perhaps this analogy doesn’t really address *using* an IDE, so much as it does learning how to accomplish a task from scratch with a complex tool, versus without it. As Nick said, start with the bare bones and then move to the more complex tools. For one thing, once you understand the underlying concepts, you are better placed to judge how useful a particular tool is, because you will be aware of a World outside its boundaries. You can assess what it *can’t* do, as well as what it can. But if your entire knowledge of the problem domain is derived from a tool, complex or otherwise, you are limited to thinking about the problem space only in terms of the tool you employed to learn it. That doesn’t mean IDEs are evil, though.

  14. eric says:

    Hey, everyone loves Clippy.

  15. Karl Sanford says:

    I wonder if both groups solving the puzzles were given adequate training on their tools. If they weren’t, then I think they are really measuring how intuitive a tool is versus how helpful a tool is for solving the puzzle. If think this is evident in the conclusion of the experiment where the helpful tool users where using trial and error to accomplish tasks.

    While I think there is an argument that a helpful tool would be even more helpful if it were also intuitive, I don’t think it is necessarily required. I would think that receiving training for a helpful tool can really make a difference.

3 Pings/Trackbacks for "Is helpful software really helpful?"
  1. [...] Is helpful software really helpful? — The Endeavour RT @rickasaurus: Seems to depend on frequency of use. “@JohnDCook: Software that tries too hard to help can harm http://bit.ly/cjK80s” (tags: via:packrati.us) This entry was posted on Wednesday, August 25th, 2010 at 1:31 am and is filed under Bookmarks. You can follow any responses to this entry through the RSS 2.0 feed. Responses are currently closed, but you can trackback from your own site. [...]

  2. [...] Um passo pra trás para dar dois passos para frente Publicado em agosto 24, 2010 por Manoel Galdino A frase acima é do Lenin (mais ou menos isso) e se referia a idéia de que às vezes é preciso recuar para conseguir avançar mais. Essa frase me veio a cabeça quando li um texto do Jonh Cook, do blog endeavour. [...]

  3. [...] Is helpful software really helpful? Ford-Chevy arguments Would you rather have a chauffeur or a Ferrari? ? X [...]