Comments on: Software sins of omission

By: ARaybould

ARaybould — Sun, 27 Nov 2011 15:15:59 +0000

Having a high percentage of errors of omission does not strike me as surprising when we start from a blank state and add code until we seem to have finished. I would think that coding by modifying or otherwise reusing existing code would lead to a higher proportion of errors of commission (case in point: the failure of the first Ariane 5 launch.) Perhaps the difference between Marick’s and Grass’ figures, if statistically significant, can be attributed to different patterns of development over time or applications.

By: Nunc dimittis — The Endeavour

Nunc dimittis — The Endeavour — Sat, 19 Nov 2011 16:58:17 +0000

[...] Software sins of omission (Software and the Book of Common Prayer) Doing good work with bad tools (Charlie Parker story) Dave Brubeck mass (Mass composed by a jazz icon) [...]

By: Brian Marick

Brian Marick — Wed, 05 Jan 2011 00:54:12 +0000

Back when I had dreams of a PhD, I did a survey of many classifications of errors. 75% is on the high side for faults of omission. That's not to say that I'm downplaying them - on the contrary, they've obsessed me for many years. But we should realize the % of faults of omission is pretty context-dependent. Regarding Jester and other weak-mutation coverage tools, I didn't find them wildly successful at signaling faults of omission. Most of the current crop of mutation tools are fairly limited in the kind of program transformations they can do. My GCT tool did a lot more, but I concluded that was a blind alley.

By: Daniel Lemire

Daniel Lemire — Tue, 04 Jan 2011 23:33:20 +0000

Ironically, here’s what I get when I visit Glass’ web page:

http://www.robertlglass.com/

“user warning: Incorrect key file for table ‘node’”

Obviously, he didn’t plan for this type of error, otherwise, we would get a user-friendly message.

By: John Davis

John Davis — Fri, 15 Jan 2010 15:24:17 +0000

Interesting posting but a slight error I think in your definition:

“The things left undone are called sins of omission; things which ought not to have been done are called sins of commission.”

Sins of comission are usually defined as things which we know are wrong but we do them anyway ! It is the knowledge of its wrongness that is the key, I’m not sure that too many software bugs ever fall into this category – well I hope not anyway !

By: Gene Harris

Gene Harris — Thu, 14 Jan 2010 04:23:26 +0000

Thanks for this posting. It reminded me of a paper I wrote for Pentasafe a few years ago: Software developers are [usually] bad software testers.

By: Filip van Laenen

Filip van Laenen — Tue, 12 Jan 2010 21:41:42 +0000

You should try mutation testing, like e.g. Heckle does for Ruby. It’s amazing how many errors it can find, both sins of omission and commission. The tool is actually pretty annoying when you start to use it (I had a lot of reactions of the type “but why would you try to do that?”) but after a while you don’t want to work without it any more. Unfortunately, it doesn’t work for user interfaces

By: David Wolever

David Wolever — Tue, 12 Jan 2010 16:38:18 +0000

The more I'm writing code, the more I believe this. I used to be skeptical of the suggestion that exceptions are bad… But, for exactly the reason you're talking about here, I've now come to see the light. This is actually one of the features I really, really like about languages which are based on pattern matching: it's a lot easier to notice when error cases aren't handled. For example, it only takes a basic knowledge of Erlang to see that:


case file:open(SomeFile) of
    { ok, File } -> print_lines_in(File).

Is very obviously missing { error, _ } error handling. But, given roughly the same Python code:


file = open(some_file)
print_lines_in(file)

It's a lot harder to see that an error handler is missing.

By: Kevin Rodrigues

Kevin Rodrigues — Tue, 12 Jan 2010 16:30:24 +0000

Considering that it is difficult to test all the logical paths that the software can take, perhaps it is better to have beta testing by the users and then note down the issues that they faced and what improvements they would want. This can be done faster by prototyping or tracer bullet method to pin down as close as possible what the users actually need i.e what all logical paths are going to be used in the field. Once this is done, making the necessary implementations, reviews and testing would be much more easier because of their concreteness.

By: John

John — Tue, 12 Jan 2010 14:18:45 +0000

Mat, I agree with you that TDD could help reduce sins of omission. It’s easier to think about what needs to be done when you don’t (yet) need to implement it. But after-the-fact unit testing won’t help as much. If you write your code first, it’s natural to see your code base as your to-do list for writing tests. In that case you’re mostly going to write tests that correspond to code you’ve written, though a few more logic paths may come to mind in the process.

By: mat roberts

mat roberts — Tue, 12 Jan 2010 13:58:02 +0000

My experience when writing code test first, is that it actually help a lot with error conditions, and other sins of omission. The process just helps me think much more clearly about what can go wrong. It also forces you to right code in such a way that error conditions can be simulated. So I don’t entirely agree with you that Unit Testing (or a least TDD) doesn’t help with sins of omission.

You are of course right though that Unit Testing won’t stop all bugs, so review and also exploratory testing are still very important.