Comments on: Maybe NASA could use some buggy software http://www.johndcook.com/blog/2009/10/08/nasa-buggy-software/ The blog of John D. Cook Thu, 11 Mar 2010 17:04:21 -0500 http://wordpress.org/?v=2.8.4 hourly 1 By: Anon http://www.johndcook.com/blog/2009/10/08/nasa-buggy-software/comment-page-1/#comment-25710 Anon Sat, 10 Oct 2009 05:22:36 +0000 http://www.johndcook.com/blog/?p=3343#comment-25710 The space shuttle software certainly doesn't sound bug free to me! The routines that control the physical systems my be bug free but the system as a whole certainly sounds like it isn't. When you have ui that tells you things are off when they are on but it is too cost prohibitive to change them, so you have 200 pounds of errata and manuals does not sounds like a resounding software engineering feat to me. http://ocw.mit.edu/OcwWeb/Aeronautics-and-Astronautics/16-885JFall-2005/LectureNotes/detail/embed22.htm The space shuttle software certainly doesn’t sound bug free to me! The routines that control the physical systems my be bug free but the system as a whole certainly sounds like it isn’t. When you have ui that tells you things are off when they are on but it is too cost prohibitive to change them, so you have 200 pounds of errata and manuals does not sounds like a resounding software engineering feat to me.

http://ocw.mit.edu/OcwWeb/Aeronautics-and-Astronautics/16-885JFall-2005/LectureNotes/detail/embed22.htm

]]> By: joel8360 http://www.johndcook.com/blog/2009/10/08/nasa-buggy-software/comment-page-1/#comment-25692 joel8360 Fri, 09 Oct 2009 17:23:07 +0000 http://www.johndcook.com/blog/?p=3343#comment-25692 "as the probability of bugs goes to zero, the development costs go to infinity. " If this is true, then you will be able to do one of the following: - beggar Knuth - establish your reputation as the man who found scads of bugs where Knuth found none. “as the probability of bugs goes to zero, the development costs go to infinity. ”

If this is true, then you will be able to do one of the following:
– beggar Knuth
– establish your reputation as the man who found scads of bugs where Knuth found none.

]]> By: John http://www.johndcook.com/blog/2009/10/08/nasa-buggy-software/comment-page-1/#comment-25674 John Fri, 09 Oct 2009 11:23:25 +0000 http://www.johndcook.com/blog/?p=3343#comment-25674 Andrew, good points. I don't disagree that perfect software is preferable to buggy software. But sometimes the realistic alternative to buggy software is no software, and sometimes buggy software is better than no software. Your point about budget surplus is right on. The savings evaporates, so you might as well spend all you can get. Andrew, good points. I don’t disagree that perfect software is preferable to buggy software. But sometimes the realistic alternative to buggy software is no software, and sometimes buggy software is better than no software.

Your point about budget surplus is right on. The savings evaporates, so you might as well spend all you can get.

]]> By: Andrew T. http://www.johndcook.com/blog/2009/10/08/nasa-buggy-software/comment-page-1/#comment-25673 Andrew T. Fri, 09 Oct 2009 11:08:01 +0000 http://www.johndcook.com/blog/?p=3343#comment-25673 Would you like it if the electronically controlled brake system on a train failed at the wrong time? Some software must be "perfect" - where human lives are at risk, this is especially true. Guess what, if you're the responsible party for software that maims or kills someone, you're liable, just like if an Engineering firm builds a structure that fails. On the point of unmanned missions. In software, as with many other places, as soon as you save money, it's not viewed as a surplus - it's viewed as a place to cut the budget. So a mission that would cost $500M is done for $375M because of slightly lower quality code (they outsourced to MIT), that other $125M pretty much disappears, you didn't spend it, you don't keep it. And anyway, I honestly don't think that $250M of $500M goes to the software on these missions, I don't even think software contributes 25% of the cost of any of these missions. A great article on the way NASA writes software: http://www.fastcompany.com/magazine/06/writestuff.html Would you like it if the electronically controlled brake system on a train failed at the wrong time? Some software must be “perfect” – where human lives are at risk, this is especially true. Guess what, if you’re the responsible party for software that maims or kills someone, you’re liable, just like if an Engineering firm builds a structure that fails.

On the point of unmanned missions. In software, as with many other places, as soon as you save money, it’s not viewed as a surplus – it’s viewed as a place to cut the budget. So a mission that would cost $500M is done for $375M because of slightly lower quality code (they outsourced to MIT), that other $125M pretty much disappears, you didn’t spend it, you don’t keep it. And anyway, I honestly don’t think that $250M of $500M goes to the software on these missions, I don’t even think software contributes 25% of the cost of any of these missions.

A great article on the way NASA writes software:
http://www.fastcompany.com/magazine/06/writestuff.html

]]> By: Joff http://www.johndcook.com/blog/2009/10/08/nasa-buggy-software/comment-page-1/#comment-25647 Joff Fri, 09 Oct 2009 00:15:11 +0000 http://www.johndcook.com/blog/?p=3343#comment-25647 If it doubles the cost to go from 98% to 99%, but if that failure happens, the cost could be in the tens of billions of dollars (and human lives), then yes, it's probably worth it. If it doubles the cost to go from 98% to 99%, but if that failure happens, the cost could be in the tens of billions of dollars (and human lives), then yes, it’s probably worth it.

]]> By: Josef http://www.johndcook.com/blog/2009/10/08/nasa-buggy-software/comment-page-1/#comment-25644 Josef Thu, 08 Oct 2009 23:36:38 +0000 http://www.johndcook.com/blog/?p=3343#comment-25644 Whenever humans and machines cooperate the human should always be the weakest link. What it means is that the machine has to be far more reliable than the human. And it's up to us as programmers to make that happen. Sure programmers make mistakes to. But we have the advantage that we can simulate our programs forwards, backward, up, down and through a model checker. We can't do that with human beings. Not yet, and hopefully never. Whenever humans and machines cooperate the human should always be the weakest link. What it means is that the machine has to be far more reliable than the human. And it’s up to us as programmers to make that happen. Sure programmers make mistakes to. But we have the advantage that we can simulate our programs forwards, backward, up, down and through a model checker. We can’t do that with human beings. Not yet, and hopefully never.

]]> By: James http://www.johndcook.com/blog/2009/10/08/nasa-buggy-software/comment-page-1/#comment-25642 James Thu, 08 Oct 2009 22:30:16 +0000 http://www.johndcook.com/blog/?p=3343#comment-25642 Your site is spitting out ugly errors if a person forgets to fill out the email Your site is spitting out ugly errors if a person forgets to fill out the email

]]> By: James http://www.johndcook.com/blog/2009/10/08/nasa-buggy-software/comment-page-1/#comment-25641 James Thu, 08 Oct 2009 22:29:33 +0000 http://www.johndcook.com/blog/?p=3343#comment-25641 Yes, I can see political reasons playing a huge factors. Do you think the government still inject funding if more and more mission failed? Yes, I can see political reasons playing a huge factors. Do you think the government still inject funding if more and more mission failed?

]]> By: John http://www.johndcook.com/blog/2009/10/08/nasa-buggy-software/comment-page-1/#comment-25629 John Thu, 08 Oct 2009 15:01:12 +0000 http://www.johndcook.com/blog/?p=3343#comment-25629 Jason: There are political reasons as well, as Norvig points out after the quote above. If you're just looking at successful missions per dollar, you might lower your quality standards. But politics isn't measured in missions per dollar. Politically, the cost of a failed mission is much greater than the benefit of a successful mission. Jason: There are political reasons as well, as Norvig points out after the quote above. If you’re just looking at successful missions per dollar, you might lower your quality standards. But politics isn’t measured in missions per dollar. Politically, the cost of a failed mission is much greater than the benefit of a successful mission.

]]> By: Jason Dyer http://www.johndcook.com/blog/2009/10/08/nasa-buggy-software/comment-page-1/#comment-25627 Jason Dyer Thu, 08 Oct 2009 14:22:10 +0000 http://www.johndcook.com/blog/?p=3343#comment-25627 I interviewed once (didn't get the job) at a company that did firmware for aeronautics. There are legal reasons why one can't just program normally as suggested above. I interviewed once (didn’t get the job) at a company that did firmware for aeronautics.

There are legal reasons why one can’t just program normally as suggested above.

]]> By: Omar Gomez http://www.johndcook.com/blog/2009/10/08/nasa-buggy-software/comment-page-1/#comment-25626 Omar Gomez Thu, 08 Oct 2009 14:09:38 +0000 http://www.johndcook.com/blog/?p=3343#comment-25626 Nice post. I've always though we Software people are too much focused on technical aspects and forgot how important economics are. Nice post.

I’ve always though we Software people are too much focused on technical aspects and forgot how important economics are.

]]>