Exogenous Shock

by F.

Following up on yesterday’s belief-system defense, and somewhat similar, is the “exogenous-shock defense,” also from Tetlock’s Expert Political Judgment.

Forecasts always make assumptions, and these are buried in tacit “ceteris paribus” clauses—Latin for “all other things being equal.” For instance, when you predict “the sun will rise tomorrow,” you are assuming a bunch of things: the sun won’t supernova, the universe won’t implode, aliens won’t steal the sun to use it to light their hydroponic ganja gardens, and so on.

These one-in-a-gazillion events are “exogenous shocks”: they are freaky happenings that come out of the blue and screw up the reasonable assumptions on which your forecasting model is based. As Tetlock notes, “[a]nything that falls outside the expert’s framework can qualify” as an exogenous shock, and hence justify predictive error. So, to invoke this defense

Look for an exogenous shock
that you—and most people—couldn’t have foreseen
and that renders your predictive framework worthless.

Here’s an example. One expert interviewed by Tetlock was “suprised by how far Gorbachev went in making concessions on arms control issues.” He justified his error like this:

I study interstate relations. I am not a Sovietologist. Has a theory of marriage failed if it predicts that a couple will stay unhappily married, the couple toughs it out for decades, and suddenly the husband has a fatal heart attack? Of course not, the failure lies in not checking with the cardiologist. Well, my failure was not recognizing how sick the Soviet state was. (131).

To paraphrase: “I couldn’t have known how sick the Soviet system was—it was one-in-a-gazillion and came completely out of the blue. So even if I was technically wrong, it’s not a big deal.”