Comments on: Amateur software

By: Kevin Webber

Kevin Webber — Mon, 15 Feb 2010 23:57:06 +0000

The lack of quality in software development isn’t a problem at all, it’s just a fact. Amateur, professional, it doesn’t really matter to me if someone was paid to do something or wasn’t, if using it is a waste of my time it’s a waste of my time. And my time is worth more to me than the up-front cost of an average commercial product (unless you’re talking about an “enterprise” license that costs six-figures).

I don’t buy commercial software of amateur quality; let the company go out of business. If I accidentally pay for commercial software and the quality sucks, I let others know about my mistake and do more homework next time. Luckily, I got in the habit of doing decision analysis research at work for software purchases, and I carried that habit into my personal life. It helps! I avoid crappy free software like the plague. When I make mistakes with choosing free software (and I have), I usually did it to myself; too eager to save a few bucks so not enough research. Live and learn. But I fanatically support professional-quality free software, because it really is where a lot of the innovation in software development is coming from.

The world isn’t going to all of a sudden start producing nothing but high-quality software (or anything else), so remember… Caveat emptor.

By: Gene

Gene — Thu, 11 Feb 2010 15:08:23 +0000

>>I’m growing increasingly frustrated with amateur software.

I think you forgot to answer your first sentence. Or I don’t think I understood if you did answer it.

By: Daniel Lemire

Daniel Lemire — Wed, 10 Feb 2010 22:25:22 +0000

I’m not too sure what there is to be frustrated about. You probably want to write a follow-up to this blog post.

By: John

John — Wed, 10 Feb 2010 18:24:49 +0000

Jeff, as to what we can do, it depends on whether we’re consumers or producers. As a consumer, I’m leery about learning software that’s going to be an ongoing time sink, whether the software is amateur or professional. So beyond just asking whether some software appears to do what you want, you have to estimate what the long term cost of using the software will be. “Cost” includes time and money.

As software producers, we can behave professionally, even if we’re not working on a professional project. We can test documentation. I’m amazed that software projects will test their software but never test their documentation. Hire someone off the street to install your software and do some simple task with it. Make a video of their experience. Or if you can’t hire someone, ask your brother in law to do it. But run your software by at least one person who isn’t intimately familiar with it.

By: Daniel Black

Daniel Black — Wed, 10 Feb 2010 18:23:32 +0000

@John

Hmm. Then I’ll concur with you on Sage. The fact that it installs without a hitch on my machine, running only Linux, only means that it’s not as frustrating as it could be.

@Jeff & @William

I guess what you’re pointing to is a gap in the measure of reliability, which is why we hear about J.D. Powers & Associates every other commercial break, and how The Deck (ad network) is built the way it is. There seems no hope in concluding other than that we’ll always have some garbage in the stream. Is there currently, potential demand for, a means of collecting reviews of various products, a standard battery of evaluative measures, or something? Like Epinions, only not horrible (irony intended).

By: William J McKibbin

William J McKibbin — Wed, 10 Feb 2010 17:57:21 +0000

I share your concerns with what you describe as “amateur” software. But, I would also voice a concern about most “open-source” software as well (though certainly not all). We are quickly arriving into a time when professionals will be held responsible for the “solutions” yielded by the software they select and employ. In such an environment, due diligence on the reliability and validity of software applications becomes a matter of survival. If I am going to bond on a software-derived “solution,” then I need to be absolutely certain that the produced results meet certain professional standards, and frankly, it is difficult to gather evidence as to the reliability and validity of software applications that are not professional produced and maintained (ironically, professionals are often reluctant to attest to the reliability and validity of open-source software). Of course, there are exceptions to my observations. However, I tend to take the reliability and validity issues very seriously in professional practice — my clients demand that from me. Thanks for the opportunity to comment.

By: Jeff Shelton

Jeff Shelton — Wed, 10 Feb 2010 17:47:40 +0000

While I understand the frustration, what is to be done about this problem? There are bad window washers, bad dog owners, and bad dentists. In most cases, I would guess that poor performance results from failing to attend to small details that are boring, unrewarding, or difficult. This doesn’t seem to me to be just a software problem, but rather a societal one. Can people be taught to care about the details?

By: John

John — Wed, 10 Feb 2010 17:13:26 +0000

Daniel, I agree that scientists need to release their code more often. Statistical analyses are often irreproducible. But I'm afraid I have to disagree about SAGE, at least on Windows. SAGE doesn't actually run on Windows, but the SAGE web site offers a few alternatives. They offer a bootable CD to run Linux, but I was unable to download the image. They also explain how to SAGE via Linux in a VirtualBox. With some effort I eventually got that to work. Maybe SAGE works smoothly once you get past the installation.

By: Daniel Black

Daniel Black — Wed, 10 Feb 2010 17:03:03 +0000

I don't have the Python chops (yet) to do anything useful with it, but Sage is a nice piece of amateur software that installs pretty nicely and is heavily documented. Actually, I've been thinking more about this lately, and about what I might want to do upon (finally) graduating (undergrad) in June. After reading Prof. Ince's column in the Guardian, entitled "If you're going to do good science, release the computer code too," I'm thinking that there is a lot of room in science for improved--to use your term--amateur software. If scientists might better apply their time to the construction and execution of experiments and measurement exercises, and especially if their code is notably fraught with bugs and poor documentation, maybe an organized cabal of programmers could shoulder that mantel. Which is a digression of sorts, but seemed at least tangentially applicable.

By: John

John — Wed, 10 Feb 2010 16:42:01 +0000

I’ve used Stellarium a little, and I agree it’s quite impressive.

I’m not so concerned with a difficult install per se, but a difficult install often foreshadows more problems to come.

By: mat roberts

mat roberts — Wed, 10 Feb 2010 16:36:09 +0000

To be honest with you, if it doesn’t install cleanly, I give up.
Scanty documentation, on the other hand, I think you just have to live with.

For a really stunning piece of open source software check out stellarium
http://www.stellarium.org/
I cannot believe the level of visual polish they’ve achieved in this thing – so not like amateur software.
I used it to locate Mars last week, then went outside my front door and saw Mars for real. You could see the reddy colour – I was quite blown away.