Monday, July 30, 2012

Skill in Prediction: Olympic Medals Edition

Not long ago I offered a critique of a statistical methodology used to predict Olympic medal outcomes. In my critique, I explained that a naive prediction -- the medals won in the previous Olympic games -- outperformed a sophisticated statistical model, as shown below.
What the graph indicates is that the statistical model had no skill -- it provided no value-added to the naive forecast.

At the FT, Martin Stabe is tracking four such models as follows:
Here I will follow along the FT medal-tracking exercise and evaluate the model predictions against the naive forecast based on the outcomes of the 2008 Beijing Olympics. My aim is to evaluate the predictions, which is a bit different that the purpose of the FT exercise, which I take is to evaluate country performance versus expectations.

However, to evaluate country performance appropriately, one needs to know the fidelity of ones expectations. Otherwise you won;t be able to distinguish a country's outperformance from just poor expectations. Stay tuned as the games progress for the quantitative analysis.


Post a Comment