Suppose I asked where you want to eat lunch. Then I told you I was about to flip a coin and asked again where you want to eat lunch. Would your answer change? Probably not, but sometimes the introduction of irrelevant uncertainty does change our behavior.
Here’s an example I’ve seen repeatedly. In adaptive clinical trials, a patient’s treatment is influenced by the data on all previous patients. It is often the case that a particular observation has no immediate impact. Suppose Mr. Smith’s outcome is unknown. We calculate what the treatment for the next patient will be if Mr. Smith responds well and what it will be if he does not. If both doses are the same, why wait to know his outcome before continuing? Some people accept this reasoning immediately, but others are quite resistant.
Not only may a patient’s outcome be irrelevant, the outcome of an entire clinical trial may be irrelevant. I heard of a conversation with a drug company where a consultant asked what the company would do if their trial were successful. He then asked what they would do if it were not successful. Both answers were the same. He then asked why do the trial at all, but his question fell on deaf ears.
While it is irrational to wait to resolve irrelevant uncertainty, it is a human tendency. For example, businesses may delay a decision on some action pending the outcome of a presidential election, even if they would take the same action regardless which candidate won. I see how silly this is when other people do it, but it’s not too hard for me to think of analogous situations where I act the same way.
I have a little more sympathy with the clinical trial example now when I wrote this post almost a decade ago.
It may be true that under the assumptions of the model a trial would treat a patient the same way regardless of an individual outcome. But it could make sense to wait to see if something outside of expectations happens, such as a more negative reaction than was thought possible.
I feel that this is a highly useful phrase for discussing all kinds of simulations with engineers and scientists. We have the computational powers to simulate all sorts of processes with basically arbitrary precision (e.g. in traffic or economics) but completely ignore system-immanent randomness or underlying motives we’ll never be able to explore. Thanks for the post!
The consultant asking why do the trial at all needs to work in some other industry.