The Sin of Certainty
In the Atlantic Lane Wallace addresses why experts seem to be wrong so much of the time –
In his new book, How We Decide, Jonah Lehrer cites a research study done by U.C. Berkeley professor Philip Tetlock. Tetlock questioned 284 people who made their living “commenting or offering advice on political and economic trends,” asking them to make predictions about future events. Over the course of the study, Tetlock collected quantitative data on over 82,000 predictions, as well as information from follow-up interviews with the subjects about the thought processes they’d used to come to those predictions.
His findings were surprising. Most of Tetlock’s questions about the future events were put in the form of specific, multiple choice questions, with three possible answers. But for all their expertise, the pundits’ predictions turned out to be correct less than 33% of the time. Which meant, as Lehrer puts it, that a “dart-throwing chimp” would have had a higher rate of success. Tetlock also found that the least accurate predictions were made by the most famous experts in the group.
Why was that? According to Lehrer,
“The central error diagnosed by Tetlock was the sin of certainty, which led the ‘experts’ to impose a top-down solution on their decision-making processes … When pundits were convinced that they were right, they ignored any brain areas that implied they might be wrong.”
I see something similar developing on the political right currently. Since late last year people have been predicting with growing certainty that the GOP is going to make big gains in the November elections, and it certainly seems that way at the moment. Numbers are trending towards the GOP. Republicans have won a number of 3 out of 4 closely watched elections recently. Obama’s popularity is dropping almost daily. All that ignores a few facts. First – it is still almost 7 months to the election. Second – Like it or not the economy is recovering. Not quickly and not as strongly as we would like but it is. Third – The worst of the predictions have not come true. The Stimulus package was passed a year ago and we don’t have hyperinflation and China hasn’t stopped buying US bonds. Fourth – Immigration reform is going to be on the table again pre-election and it is going to hurt the GOP gain badly. And Fifth – The GOP remains wildly unpopular and appears very ineffective based on it’s handling of Michael Steele and other such problems. Those issues can be mitigated but they have to be addressed. So far that doesn’t really seem to be happening – everyone appears content to try and ride Health Care Reform to victory. I don’t think that is going to happen. 7 months is a long time to stay mad. especially when so far the impact from HCR has been minimal.
Of course I am not an expert so my prediction probably has a much less than 33.333% chance of being correct.