From Russ Roberts on the latest EconTalk podcast:
… this is really embarrassing as a professional economist — but I’ve come to believe that there may be no examples … of where a sophisticated multivariate econometric analysis … where important policy issues are at stake, has led to a consensus. Where somebody says: Well, I guess I was wrong. Where somebody on the other side of the issue says: Your analysis, you’ve got a significant coefficient there — I’m wrong. No. They always say: You left this out, you left that out. And they’re right, of course. And then they can redo the analysis and show that in fact — and so what that means is that the tools, instead of leading to certainty and improved knowledge about the usefulness of policy interventions, are merely window dressing for ideological biases that are pre-existing in the case of the researchers.
Related post: Numerous studies have confirmed …
15 thoughts on “Window dressing for ideological biases”
Not knowing the context of the quotation, I’m only 90% confident the following link is relevant (it’s about finance, rather than economy)… Yet, I think it is an interesting one anyway so: Quants: The Alchemists of Wall Street.
1. Economics has never been a normative discipline. It’s about what happens if you do X, not whether you should do X. (e.g., implement price controls).
2. There are competiting objectives (employment levels versus inflation). You can’t have everything. Gain something and lose something.
3. Along w Smith, Keynes etc. Karl Marx is an important figure in economics. That should tell you a lot about the diversitivity of opinions in the field.
The old joke: Ask 2 economists for their opinion and get three answers.
James: One problem is that the normative biases of the researchers blur the descriptive goals of the discipline. Another problem is that multiple regression is inadequate to the task of teasing apart countless confounding causes and effects.
Thanks John for bringing Russ Robert’s excellent comment to our attention! I completely agree that it is “window dressing” for ideological biases. In no way can econometrics explain or encompass the immense complexity of our social world – plus there are always black swans, so that the Quants Alchemists of Wall Street will always get it wrong over the long term…and sometimes even over the short term! You never know when a tsunami is going to hit you, ask the Japanese!
This said, some ideologies are better rooted in reality than others – hence their success. Just to explain what I mean: consider Karl Marx and Keynes. Marx’s ideological construct (communism) was obviously relevant to the social tensions and extreme poverty and suffering caused by the industrial revolution. In that context, socialism and communism thrived. Keynes was highly relevant to the Great Depression and gave a boost to the role of the State in the economy (to jumpstart it when the private sector is down) . Today many don’t see Keynes’ relevance to the current Great Recession and cling to Milton Friedman’s teachings, hoping unfettered capitalism will bring us out of the recession.
Who’s right? In my opinion, the only meaningful way for the debate to proceed is to review the conflicting ideologies in order to determine which one is more solidly anchored in the social reality.It’s not a question of mathematics…Social issues never are!
@James You are too nice to economics as a science with 1. Economists do help set policy as in they get pulled in by the opposition party to say “here is why you are screwing up and why we need better policies like the ones that the party paying our paychecks claim they’ll do”. Then whenever anything goes wrong it isn’t the model that was wrong but “some unexpected think I couldn’t possibly be expected to have included in my model”.
The book Black Swan rips into this kind of thinking pretty good. Essentially people make assumptions so they can come up with a model they can understand then quickly forget their assumptions. They go for a period of time assuming their model must be right since it hasn’t broken yet and then when something happens that breaks the model they throw up their hands and say “not my fault”. It is their fault because claiming to be able to model something that includes unpredictability (pretty much anything not pure physical science) while not insuring against the edge cases you ignored is unprofessional at best and likely immoral in others (like bank CEOs gaming compensation by assuming nothing ever bad will happen when they play shell games with themselves, or US automotive always just needing this one more bailout for the last 30 years while still paying execs good money for “performance” in good years).
“[Manzi] argues for humility and lowered expectations when it comes to understanding causal effects in social settings related to public policy.” Unfortunately, success in economics (and politics) is directly proportional to ego.
Stop — you’re both right!
Economics is a floor wax and a dessert topping!
But seriously, folks: what are we to make of a discipline in which the Nobel Prize has gone to Paul Samuelson, Friedrich Hayek, Milton Friedman and Paul Krugman? The problem of two-handed economists will be with us always.
 Oh, right, the Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel. Excuse the hell out of me.
 Neglecting, of course, the revered Hyman “Lefty” Minsky.
I’m in the middle of Daniel Kahneman’s (fabulous) book Thinking, Fast and Slow, and he talks at some length about the mechanisms that promote this behavior. Stop bashing economists; it’s a universal behavior. Interestingly, you can subvert the bias (to some extent) by presenting individual cases that violate the bias before presenting the general model.
Disclaimer: I am not an economist, though my former boss wished I were.
Dave: I agree it’s universal behavior.
I’m not bashing economists, I’m quoting an economist.
John: My apologies, I spoke badly. I felt that others in the thread were attributing the behavior to economists specifically; I did not mean to imply that you had said anything of the sort in your original post.
Dave: No worries. Economists do get more than their share of criticism. I suppose because laymen can tell when they’re wrong, more so than some other fields.
John, now you’re being too nice to economists. It’s not that only economists have this kind of problem, and not just that laymen can tell when they’re wrong more than in other fields. It’s that people actually listen to them when they make ridiculous claims based only on ego and ideology but dress them up as scientific fact with econometric window dressing.
When some sociologist makes a ridiculous grand causal claim about some social question based on a regression or two, few people pay attention, and very rarely are there any real world consequences that harm anyone. But a mass proportion of economists are ideological charlatans who buttress ideological claims with claims that the political goals of said ideology are scientifically proven to be “right”.
Sure, maybe the circus magician and the snake oil salesman are both selling lies for personal gain. But if you think the magician has true powers to make little things disappear from his hand and teleport into his pocket, you just have a silly inconsequential belief. If you have cancer and buy snake oil instead of going to a real doctor, you die. Seems obvious the snake oil salesman deserves ridicule and criticism but the circus magician does not.
I have to agree with you. Economists spew out ideas and the governments pass laws/spending bills based on them. If the theories/recommendations don’t pan out I suggest economists should be subject to malpractice just like engineers doctors etc.
Second: economists getting “Nobel” prizes for theories that contradict each other is crazy. When both the super nutty conservatives and the super nutty liberals can both point at Nobel prize winners to justify their plans something is wrong.
Economics is a very complicated field where it is pretty much impossible to study things in isolation (similar to medicine) however, often a single comparison is used to “prove” an idea. “Brazil defaulted on their sovereign credit and they are fine therefore we can do it too” kinds of arguments. No sort of reasoning like is it moral to spend the money giving your citizens full healthcare and 8 weeks vacation a year and then refuse to pay the bill. No others did it so it is only fair we can do it.
If I recall correctly this is from a show with Robin Hanson. Just a couple things to note (as someone who’s listened to ~50 EconTalk shows):
• Russ has developed a greater and greater aversion to maths over the show’s life. But I think that partly reflects his innate bias–he was never a Geanakouplos style mathematical renovator of the dilapidated Arrow-Debreu drawing room.
• If I recall correctly this is from a show with Robin Hanson. Hanson studied bias in health economists. A major result is that no one changed their opinions despite statistical evidence.
• Again, if memory serves—the discussion went on to talk about kinds of evidence that do persuade, and I think simple plots of new data was seen as persuasive. (My own thoughts may have mixed into my memory of the show at this point. But I would add something about compelling narrative — see http://thonyc.wordpress.com/2011/06/22/but-it-doesn%e2%80%99t-move/#comment-1791)
• I think it’s fair to say that neither Russ nor Robin see as making the situation worse as such. Faintly stinky regressions are par for the course, but only to the extent of overconfidence in them are they bad.
• The main point was that fancy statistics simply fail to convince anybody. I would analogise this to how, in economics, psychology, and many other fields, it’s tough to find that crucial experiment—one that will definitively kill an opposing viewpoint because it’s just impossible to go on believing
Xafter seeing the evidence. It’s usually possible to maintain one’s viewpoint and simply say “Well, they didn’t test it the right way”.
Oops. Meant to add on the first bullet that Russ has long been of the mind that “Economics is not a science and should not pretend to be” (even more so after 2008). He wrote a bit of didactic fiction to get his viewpoint across.
(I would argue that the honest straightforwardness of didactic fiction is laudable, but there are laudable aspects to using mathematical models as well.)
My own view is something like what P Krugman said about maps of “deepest darkest Africa” (i.e. the interior). Economics, like any field of inquiry, is capable of being more or less scientific, taking missteps, embracing or discarding mistaken views, and so on. The field is certainly not finished but it makes sense to try to be scientific (and to understand what that really means).
It’s imperfect but that’s not to say in 200 years the field will be just as badly off as it currently is. It depends how good of a job present-day economists do.