No one cares about statistical results per se. They care about what they should do in response to the results. The purpose of statistical analysis is to inform decisions, and yet this purpose can be obscured by point estimates and confidence intervals around the values of Greek letters.
We are all forced to make decisions under uncertainty. It would be preferable to remove all uncertainty, though that’s not usually an option. By the time the fog of uncertainty has lifted, it may be too late to act.
“Certain knowledge, to the extent that it ever comes, is given us only after the moment of opportunity has passed.” — George Gilder
Since we have no choice but to make decisions while things we’d like to know remain uncertain, we must work intelligently with uncertainty. Too often people use statistics as part of a two step process. First they estimate some quantity statistically, along with some measure of uncertainty. Then forgetting that the uncertainty exists, they move on to making decisions. Decision theory carries the estimation uncertainty forward and uses it to recommend a course of action, not a set of parameter estimates.
Decision theory considers the utility of possible outcomes, and recommends the course of action that optimizes the expected utility. Traditional statistics optimizes a utilities as well, though it chooses utilities for mathematical convenience, not customized to a particular problem. This may be a reasonable generic approach, but it is possible to make better decisions using utilities that reflect your unique costs and benefits.
Traditional statistical methods aim to minimize the squared error in estimating parameters. This can be less than ideal for several reasons. First of all, maybe you don’t care directly about model parameters themselves, but other things that you calculate based on these parameters. Second, squared error implicitly assumes that the costs of being wrong are symmetric, that under-estimating and over-estimating carry the same consequence. This is seldom the case in application. For example, having one more unit in inventory than needed may lead to extra storage costs or taxes. Having one less unit than needed means a lost sale. Why should these costs be the same?
Finally, squared error is unnatural in many applications. The cost of lost sales, for example, is proportional to the number of sales lost, not the square of the sales lost. A realistic model of costs could take other forms as well. For example, maybe the cost of error has some maximum value; if you’re wrong by more than a certain amount, it may not matter just how far off you were.
If you would like help using your data to make better business decisions, please call or email to discuss your project.
Trusted consultants to some of the world’s leading companies
