NCAA Brackets: Why Skillful Prediction is So Difficult

Last week I explained that Nate Silver, the widely admired NY Times blogger and political prognosticator, has been unable to demonstrate an ability to skillfully prediction the NCAA basketball tournament over the past three years.

Silver's quantifications did put Louisville as the most likely team to win the tournament -- which they did -- but that was hardly a skillful prediction as Louisville was tabbed by the NCAA selection committee as the overall number 1 seed. You wouldn't have needed any fancy methodology to pick Louisville, as they were the obvious and naive choice.

Writing at The Sports Economist blog, Victor Matheson provides some data to help explain why it is so difficult to show skill in predicting the NCAA Tournament:

Seed	Men’s Tournament, 1985-2011
	Win Margin	Win %
1	25.84	100.0
2	16.77	96.2
3	11.53	84.6
4	9.49	78.8
5	4.54	66.3
6	3.94	68.3
7	2.20	58.7
8	-0.16	48.1

He explains:

[H]istorically, the people at the NCAA who put together the brackets have done a remarkably good job seeding the teams. Obviously, there is always uncertainty of outcome in any sporting event – that’s quite a bit of the allure of spectator sports – but higher seeds tend to do better than lower ones and beat lower seeds by higher margins. Here’s the data since the men’s tournament went to a fully seeded 64-team tournament (minus the last couple of years I haven’t gotten around to updating yet.)

What the data show is that if you had simply picked the higher seed to win each game in the first round of 32 games, you would have achieved a greater than 75% success rate in predicting winners. Beating that level of forecasting is difficult, obviously.

It is no different for weather forecasters and mutual fund managers -- As Neils Bohr once said, prediction is hard, especially so when it is about the future.