Saturday, May 12, 2018

Reverse Engineering Bermon & Garnier (2017)

Last week, Ross Ticker, Erik Boye and I wrote a letter to the British Journal for Sports Medicine calling for the authors of Bermon and Garnier (2017, BG17) and their sponsor IAAF to release the performance data used in their study. You can read our letter here.

Professor Joe Guinness, a statistician and visiting assistant professor at Cornell (@joeguinness) has attempted to reproduce the reported performance results in BG17 for the women's 800m, which we discuss in our letter.
BG17 report an average time of 121.80 seconds with a standard deviation of 5.42 seconds, for 64 times included in the analysis. Prof. Guinness sought to reporduce these numbers by brute force (his code is linked in the Tweet above).

He has found that he can only come close to reproducing the times by removing Caster Semenya's 2011 time plus that of one other athlete. See his results above. He notes in a Tweet: "There are some caveats here, especially how rounding is dealt with, so this shouldn’t be taken as definitive."

If these numbers are correct then it would mean that Caster Semenya's time was removed while 2 times from Mariya Savinova in 2011 and 2013 would remain. Savinova's times have officially been removed from the IAAF database after she was suspended for doping at both the 2011 and 2013 World Championships.

The inclusion of Savinova alone would call into question the meaningfulness of BG17, and the deletion of Semenya's time would be curious. Of course, we cannot be sure about any of this until IAAF and BG17 release their data.

The longer the stonewall the more questions will be raised as to why the just don't release the data. Are there some things in their work that they are afraid to show?


Post a Comment