Let’s rehash. There was something on the order of an 0.5% chance the Red Sox would blow their nine-game lead over the Rays, and a 2% chance the Braves would lose their seven-and-a-half-game edge over the Cardinals. Then in tonight’s matchups, there was a 13% chance the Phillies would come back to beat Atlanta, a 5% chance the Orioles would come back to beat Boston, and an 0.3% chance the Rays would come back to beat New York. Multiply it all out, and the odds of witnessing what we just witnessed were worse than one in 500m.Writing at the FT, Jason Abbruzzese compares such numbers to the numbers that helped to create the global financial crisis:
The statistical revolution of the last 50 years has seriously impacted both the financial markets and professional sports. The Red Sox are among the teams that have attempted to most directly embrace this numbers revolution – often called Sabermetrics, which was written about by Michael Lewis in ‘Moneyball’ – in theatres now.But even though we know this to be the case, in the aftermath of witnessing an event that not long ago we would have dismissed as sheer fantasy, we reinterpret experience in a way that transforms the unexpected into a narrative of cause-and-effect that now makes any other outcome seem improbable from the vantage point offered by hindsight. Again, Abbruzzese:
In finance, computer-based trading allowed the best and brightest to set their minds to figuring out just which algorithm can take advantage of increasingly complex financial instruments.
Looking back on each of these situations, we see that these numbers are often only as good as the people they’re built on and the people who built them. Statistics didn’t predict the fatal flaws in CDOs. Indeed, it was a formula that felled Wall Street.
Statistics did not predict the history-making crash of the Boston Red Sox – widely regarded before the season by statisticians and overweight sportswriters alike as the best team in baseball.
Nor could they. Statistics by their nature are backward-looking and can only give us an idea of what to expect.
Down to their final out, Rays batter Dan Johnson, batting an anaemic .119 (that’s bad, cricket fans), with two strikes, launched a ball out of the park that tied a game that the Rays would go on to win.This dynamic is of course illustrates what Nassim Nicholas Taleb calls a black swan:
The chances of that are small, to say the least, yet not a single person in Red Sox nation looks back at it with surprise. And that’s exactly how many of us look back at the financial crisis – it was inconceivable, but now none of us are surprised.
I stop and summarize the triplet: rarity, extreme impact, and retrospective (though not prospective) predictability. A small number of Black Swans explains almost everything in our world, from the success of ideas and religions, to the dynamics of historical events, to elements of our own personal lives.