One advantage of crude models is that we know they are crude and will not try to read too much from them. With more sophisticated models,
… there is an awful temptation to squeeze the lemon until it is dry and to present a picture of the future which through its very precision and verisimilitude carries conviction. Yet a man who uses an imaginary map, thinking it is a true one, is likely to be worse off than someone with no map at all; for he will fail to inquire whenever he can, to observe every detail on his way, and to search continuously with all his senses and all his intelligence for indications of where he should go.
From Small is Beautiful by E. F. Schumacher.
Obviously crude models are not always better. But I like to have some evidence that a complex model is worthwhile before I invest too much effort in it. And I’m well aware of forces that reward complexity for its own sake.
Update (15 May 2015): From Simple Rules by Donald Sull and Kathleen M. Eisenhardt:
We often assume that the best way to make a decision is by considering all the factors that might influence our choice and weighing their relative importance. Psychologists have found, however, that people tend to overweigh peripheral variables at the expense of critical ones when they try to take all factors into account. … Simple rules minimize the risk of overweighing peripheral considerations by focusing on the criteria most critical for making good decisions.
6 thoughts on “Advantages of crude models”
“All models are wrong; some are useful.” Albert Einstein
The advantage of technology is that we can create and solve more and more complicated models. The problem is that people forget that the models are still wrong. I’ve seen greybeard engineers disprove months of computer work using a pencil and paper.
When we train new apprentices (structural analysis of critical airplane components) we require that they come up with some crude models to put an upper and lower bound on the answer before reviewing their computer solution.
I think it is the same in quantitative finance: better use Black Scholes and have a working risk management which plans for the worst than having a super sophisticated model which nobody understands and betting your last shirt on it.
In my experience people forget really quickly that the models are crude and do start to read too much from them. I found that once I developed quick and dirty models (thinking ‘they should be used only this year until we get something better’) they are still being used almost ten years later. Scary.
“All models are wrong; some are useful” is by George Box, not Albert Einstein.
I’ve thought about this particularly with regard to Kaggle, Netflix Competition, etc. Pragmatic Chaos & BellKor squeezed the lemon nearly dry, but did they provide real insight into the problem of movie recommendations?
If you look at the leaderboards on Kaggle, you’ll usually see that all the top scores are razor thin margins on each other. Rarely do you see a jump-like improvement.
Regarding the quotes of Box and Einstein given in this thread … Einstein is often attributed as having said “All models should be made as simple as possible, but not simpler.” However, when I tried to find out when/where he said this, it turns out that maybe he never said it at all, despite its frequent attribution to him. But it is a great quote about modeling, nonetheless! — even if Einstein is not the author!