Daniel Davies (via Brad Delong) in a classic blog post explained his reasoning for calling bullshit on the Iraq War in the summer of 2002:
Anyway, the secret to every analysis I’ve ever done of contemporary politics has been, more or less, my expensive business school education (I would write a book entitled “Everything I Know I Learned At A Very Expensive University”, but I doubt it would sell). About half of what they say about business schools and their graduates is probably true, and they do often feel like the most collossal waste of time and money, but they occasionally teach you the odd thing which is very useful indeed.
- Good ideas do not need lots of lies told about them in order to gain public acceptance.
- Fibbers’ forecasts are worthless.
- The Vital Importance of Audit.
I did not have an MBA but my graduate work was in a similar field where the entire point of attending that school was to drink really good beer and have the phrase “WHAT IS THE COUNTERFACTUAL” beat into your head until you have very odd dreams concerning low probability counterfactuals.
When you want to know the causal effect of an intervention (policy change, medical treatment, whatever) on something, you need to compare two states of the world: the world in which the intervention occurred and the world in which it did not. The latter is the counterfactual world. Since most of us only get to live in one world (most of the time), observing the counterfactual is a rather tricky thing to do. Of course, there are various worthy techniques….
What we really want to know is how the world is different due to the intervention and only the intervention….
The most important point is that almost nobody is explicit about this in policy debates, even when the policy is crucially important. Will health reform cause employers to drop coverage or not? Well, we only have one world. The counterfactual needs to be constructed. It can’t simply be assumed to be the pre-reform world, because employers have been dropping coverage for years. There’s a trend. Other things may change (the economy, the nature of the labor market, etc.), so we’d want to control for those. And so forth.
This is worth thinking about. Next time you’re involved in a policy debate, ask your opponent, what (s)he is taking as the counterfactual? If (s)he doesn’t even know what you’re talking about, you’ve already won, even if (s)he won’t admit it.
This is extraordinarily egg-headed. It is extraordinarily valuable to know what counterfactual people are talking about and arguing forth. However, on the political level, it is an impossible argument to make. We know for a damn near good fact that ARRA/stimulus was a success when compared to a counterfactual world of no ARRA/stimulus. It probably kept the US out of a Depression and only in a nasty recession. However the American public rightly decided that things still sucked in 2010 and blamed the party in power by giving power to arsonists, nihilists, and idiots. Counterfactual arguments are needed for policy purposes but they suck for political purposes.