When we make one part of our lives safer, we tend to take more chances somewhere else. Psychologists call this tendency risk homeostasis.
One of the studies often cited to support the theory of risk homeostasis involved German cab drivers. Drivers in the experimental group were given cabs with anti-lock brakes while drivers in the control group were given cabs with conventional brakes. There was no difference in the rate of crashes between the two groups. The drivers who had better brakes drove less carefully.
Risk homeostasis may explain why dynamic programming languages such as Python aren’t as dangerous as critics suppose.
Advocates of statically typed programming languages argue that it is safer to have static type checking than to not have it. Would you rather the computer to catch some of your errors or not? I’d rather it catch some of my errors, thank you. But this argument assumes two things:
- static type checking comes at no cost, and
- static type checking has no impact on programmer behavior.
Advocates of dynamic programming languages have mostly focused on the first assumption. They argue that static typing requires so much extra programming effort that it is not worth the cost. I’d like to focus on the second assumption. Maybe the presence or absence of static typing changes programmer behavior.
Maybe a lack of static type checking scares dynamic language programmers into writing unit tests. Or to turn things around, perhaps static type checking lulls programmers into thinking they do not need unit tests. Maybe static type checking is like anti-lock brakes.
Nearly everyone would agree that static type checking does not eliminate the need for unit testing. Someone accustomed to working in a statically typed language might say “I know the compiler isn’t going to catch all my errors, but I’m glad that it catches some of them.” Static typing might not eliminate the need for unit testing, but it may diminish the motivation for unit testing. The lack of compile-time checking in dynamic languages may inspire developers to write more unit tests.
See Bruce Eckel’s article Strong Typing vs. Strong Testing for more discussion of the static typing and unit testing.
Update: I’m not knocking statically typed languages. I spend most of my coding time in such languages and I’m not advocating that we get rid of static typing in order to scare people into unit testing.
I wanted to address the question of what programmers do, not what they should do. In that sense, this post is more about psychology than software engineering. (Though I believe a large part of software engineering is in fact psychology as I’ve argued here.) Do programmers who work in dynamic languages write more tests? If so, does risk homeostasis help explain why?
Finally, I appreciate the value of unit testing. I’ve spent most of the last couple days writing unit tests. But there is a limit to the kinds of bugs that unit tests can catch. Unit tests are good at catching errors in code that has been written, but most errors come from code that should have been written but wasn’t. See Software sins of omission.
I have a brilliant idea. If all that is needed is motivation- if all we need to do is to make the cost of making mistakes high enough that the programmer will stop making them- then why don’t we do the following: wire a shot gun into the back of every programmer’s chair, so when a bug occurs, bang. This should provide enough incentive for the programmer to just stop writing those stupid bugs in the first place.
This sounds silly, even ludicrous. But it’s where you logic goes. If making things more safe simply makes people less cautious, than making things less safe makes them more cautious.
The reality is that accidents happen anyways, the question is what happens when they do? A better example than anti-lock breaks might be seat belts- yes, they may also encourage riskier driving, but they also greatly lower the costs of accidents- both the new accidents due to riskier behavior and the accidents that would have happened regardless.
Another point I’d make is that there is a huge difference in this debate on what you consider the “static typing” side of the equation. If you’re comparing against Java/C#/C++, then you have something of a point- in these cases, you’re looking at anti-lock breaks that fail on a regular basis. These languages require you to subvert the type system regularly to perform necessary tasks. On the other hand, if the static type side is represented by Haskell/Ocaml/SML, then the debate changes dramatically.
“Risk homeostasis may explain why dynamic programming languages such as Python aren’t as dangerous as critics suppose.”
Would love to read the study data for this. Where can I find this?
And at least some people don’t think this is the case with Rails. A consultant writes:
“[Testing] This should probably be higher since it’s one of the most annoying things about picking up an existing codebase. A lot of the code we get isn’t well tested, or even tested at all.”
http://codemonkeyism.com/the-unit-testing-lie-aka-dynamic-language-lie/
But as I’m not aware of a systematic study on this in our industry, if you have data I would be really interested.
Cheers
Stephan
http://codemonkeyism.com
But did the study report on how productive the two groups of taxi drivers were? Perhaps the taxi drivers with better brakes drove faster and earned more money.
There is a point I didn’t make that I would like to make- I’d like to propose that the “safety difference” between, say, Java and Python isn’t as large as people might suppose because Java is a much more dynamic language than people suppose. Java’s static typing is, in some ways, the worst possible combination- static typing is primarily enforced only in the simple cases, which are unlikely to be wrong, and it imposes a fairly hefty cost (in terms of pointless ceremony) in checking these simple cases. But in those cases where things are much more likely to go wrong (especially in dealing with complex data types or more complex constraints) then the type system breaks down, and the programmer is forced back to the dynamic type system.
It not unlike having anti-lock breaks that work fine on quiet residential streets, but which disable themselves once you approach a busy intersection. Such breaks might very well be effectively worthless. But this isn’t a critique of the effectiveness or safety of anti-lock breaks in general, just a critique of a particularly broken implementation.
I know that I systematically write tests (sometimes with a poor coverage) when programming in C++, but not in Java. Clearly, that’s because C++ scares me into writing tests!
I would be curious to know how many people write unit tests for ECMAScript.
Maybe the presence or absence of static typing changes programmer behavior.
“A major drop, however, in the overall accident rate occurred in the fourth year as compared with the earlier three-year period. The researchers attributed this to the fact that the taxi company, in an effort to reduce the accident rate, had made the drivers responsible for paying part of the costs of vehicle repairs, and threatened them with dismissal if they accumulated a particularly bad accident record.”
As you’ve decided to focus on programmer behavior why have you ignored this lesson from the Munich study?
@Sol 2) On the other hand, a lot of nasty cruft like the Visitor pattern is completely unneeded in a dynamically-typed language with decent multi dispatch.
The Visitor pattern is also unneeded in a statically type checked language with decent multi dispatch.
Two observations inspired by a discussion on Twitter yesterday:
1) Static typing is a huge performance win.
2) On the other hand, a lot of nasty cruft like the Visitor pattern is completely unneeded in a dynamically-typed language with decent multi dispatch.
@Isaac: Interesting point. I’m not familiar with the taxi study in detail. I’m only using it as a metaphor.
The general principle of risk homeostasis seems quite plausible to me. If we make one part of our lives safer by a certain amount X, it doesn’t follow that the total risk in our lives goes down by X. We tend to “spend” at least part of that savings. Maybe not all of it. Maybe our overall risk does go down, but not by the full amount X.
@John I’m not familiar with the taxi study in detail.
Me neither – I just followed the link from Wikipedia.
@John I’m only using it as a metaphor.
Well use it as evidence of human behavior!
I read of a failed project that failed because the unit tests were just stubs which always answered true – the mythic status of that tale doesn’t detract from the lesson, we do what we have an incentive to do.
I agree that projects can abuse unit testing. They can have a sort of “cargo cult” software development process: successful projects have unit tests, so if we have unit tests we will automatically be successful (even if we don’t understand the substance of unit testing).
Here’s something I wrote on Stack Overflow in response to the question of whether a project can have too many unit tests:
@John I agree that projects can abuse unit testing.
What I’m trying to say is much more on point with the taxi driver example you put forward – until there’s a substantial risk felt by the programmer there’s no incentive for them to be less risky.
(The particular technology that might have been used to reduce risk is somewhat beside the point – although evading static type checking in a statically type checked language quickly becomes more work than going with the flow.)
i think people should start making a distinction between user-specified static typing and compiler-inferred static typing. it is the tediousness of the former that is the problem, not the captured knowledge (which is always useful to compilers).
@Mark: Good point. Statically-typed languages like C# have type inference that makes the type system less burdensome than say C++.
I agree, though I’ve arrived along a different path. If you want to call it risk homoestatis, I guess you’re free to. I call it have a developer that is not asleep at his keyboard. Any developer worth his salt will adjust his habits to fit the language he’s in, not necessarily because it’s riskier to develop in with dynamic typing, but just because the dynamic typing scenario is a different paradigm.
The important thing is that the language’s choice of static vs dynamic isn’t what’s important, it’s the developers and how skilled and flexible they are!
To a first-order approximation I think the risk-homeostasis idea is exactly right — and it’s a good thing! Every software project has some assurance level it needs to meet (which is higher for space shuttles than for roll-over webpage animations). If a well-designed static type system can help us reach that level with less effort than if we had to make do with unit tests, that’s great.
The reason I think it is only a first-order approximation is that I think the utility software users place on reliability etc is not a perfect step-function, where the program is either good enough or it is not. If we lower the price of software assurance (for instance by creating more convenient type systems, or better tools for managing unit test suites), I think customers are willing to buy more of it.
Testing and static type-checking are often presented as two sides of a spectrum, leading to the belief that more of one means you can get by with less of the other. Taking the idea to its limit, some testing advocates believe that, with adequate testing, static type-checking becomes redundant.
This belief is false, however, because there are important aspects of quality and correctness that cannot be practically tested for but can be verified by a sufficiently powerful type system. Take security, for example. Using types, it’s straightforward to make the compiler prove that code is free of cross-site scripting holes. But how would you test for the same property?
On the other hand, there are many properties that are easy to test for but impractical to encode into types.
In truth, then, testing and static type-checking are more complementary than supplementary. If you’re writing code and not taking advantage of both, there’s a good chance you’re overlooking some aspects of quality and correctness that matter.
What’s interesting about all this is that different programming tribes seem to have settled at different spots along this fictional spectrum, and, now committed, they defend their turf vocally in proportion to how extreme it is. Few middle grounders, for example, seem compelled to write about how both testing and static type-checking are useful, arguably necessary, tools that we would be poorer without. (I guess that’s why I’m writing this comment: to offer a defense for the middle ground that’s overlooked in the types-versus-testing debate.)
For me, the psychological aspect that is most interesting is that the more extreme tribes have latched onto the belief that what’s on the other side of the “spectrum” isn’t worth having. This belief, though, is readily disproved. That the tribes don’t test their beliefs and find them lacking, then, argues that identifying with one’s tribe is important among programmers, perhaps more so than the pursuit of good work.
If my SML code compiles I know a lot about it. I gain quite a lot by the time the SML code compiles. If my C++ compiles, I don’t know a lot. I know the semantic check had no problem but a lot of that nasty runtime stuff is really undefine. In SML there are only a few ways out: array and array indices, IO, references and state. It is often quite explicit and I can run off there. In C++ when I reference an array with an invalid index can potentially be writing code.
I also recognize that REAL static typing is a straight jacket. Often I could get away without it and optimistic programming (dynamic typed languages) would’ve worked just fine.
I usually get burned in both C/C++ and Perl and python by undefined values and nulls. In SML I can ban those and gain quite a bit from it. But then I have to operate in a world where nothing is a very explicit and special case.
I don’t consider C/C++/Java as really statically typed. Void *, casting and how C handles Enum discounts C. C++ suffers from similar problems but is a bit stronger. And Java views everything as an object or null so that’s a big nasty problem too.
SML’s straight jacket doesn’t replace unit testing. I can be typesafe and still be wrong ;)
@Brain Hurt:
If I had a shotgun wired as you suggest, I wouldn’t dare program with anything less than a theorem prover. The only errors left are specification errors, which aren’t bugs so much as communication breakdowns or changing requirements, and thus I am safe from getting my head blown off.
We know of no technology that can eliminate specification errors, though the requirement to rigourously express and verify all specification requirements in a formal language will quickly point out any contradictory or poorly specified requirements.
The idea of risk homeostasis is interesting but I think that it has more to do with how easy it is to write tests in dynamic languages. Specifically, no compiler errors during TDD is nice but also because your productivity is up you have more time and motivation to write tests.
You’ve explained it so well!
My development behavior also changes between statically-typed and dynamically-typed languages, but haven’t been able to explain it before.
One of the benefits of static type checking is that it makes it easier for me to keep track of parameters, in which case it’s not necessarily replacing unit tests so much as documentation (and I don’t find that more dynamic languages to be better documented — only that when I look at programs in dynamic languages, I *want* better documentation).
For this reason I’m not terribly keen on type inference. It eliminates one of the key benefits of static typing without providing the commensurate benefits of dynamic typing. It’s the worst of both worlds.
I’ve generally been underwhelmed by languages with very powerful type systems, as they often have the same problem as those languages that rejoice in type inference — I end up having to spend a fair bit of time and effort thinking about the types instead of thinking about the program.
So as you might expect, I’m unimpressed with the arguments of the extremists. I think a lot of people are, but there’s not really much you can write about being unimpressed with extremists. It just doesn’t provoke the same passions, after all.
Note, when I’m writing code, I chafe somewhat at static typing. And when I’m reading code, I chafe somewhat at dynamic typing, or dynamic tricks.
I know that when I write (or read) unit tests in dynamic languages, such as Ruby, the tests tend to be for type contracting. I don’t have to worry about this in static languages, so I can focus on “semantic” tests.
Static type checking makes property based testing (quick-check and alike) way easier which is in many cases superior to unit testing.
Hmm, there are strategies to turn unit tests into types by looking at the domain of the inputs vs outputs of your tests. One strategy assumes that if you have an int in your output you really want an int, one strategy assumes that you wanted a number, usually strings mean you want anything that can be shown. It would be nice if it gave you warnings and good defaults for how it can turn tests into types.