Before getting to the results, let me explain the methodology used to evaluate the predictions. I identified the final standings at the end of the 2010-2011 season as the baseline evaluation of skill. Skill refers to the ability to out-perform a naive baseline. The choice of naive baseline is arbitrary. Specifically, for each place in the table, I take the difference between that place and a predicted place and square it. So if you predicted Man City in 4th and they are in 1st, then the difference is 3, and squared is 9. I then add up the squared differences for all 18 places, and then take the square root of that sum. That gives a measure of the total error in the prediction.
Here are the results at mid-season:
keeperusa | 17.0 |
Arthur | 17.3 |
n-g | 17.5 |
Max | 19.0 |
Roger | 19.4 |
f.a.e | 19.5 |
Elijah | 20.1 |
itzik | 20.5 |
SKILL | 20.6 |
Unknown | 22.0 |
CNY Roger | 24.2 |
You'll note that the standings are tightly packed and that the threshold for skill appears more generous than the Bundesliga contest, which makes sense. n-g won the contest last year and is putting in a good showing at mid-season, but keeperusa and Arthur leading the way.
Right now Bolton, Norwich and Blackburn account for the greatest errors in the predictions. Stay tuned!
No comments:
Post a Comment