Monday, July 10, 2017

A Review of Bermon and Garnier 2017 (the new IAAF T Study)

Here are some comments on Bermon and Garnier (2017), which is the new study of the effects of testosterone levels of female elite athletes, commissioned by the IAAF in the aftermath of the 2015 CAS decision on Dutee Chand.

The paper is:

Bermon, S., & Garnier, P. Y. (2017). Serum androgen levels and their relation to performance in track and field: mass spectrometry results from 2127 observations in male and female elite athletes. British Journal of Sports Medicine. (available here non-paywalled)

These comments are in the form of bullet points, more or less following the flow of the paper:
  • The paper opens by discussing testosterone as something abused by athletes, especially female athletes. This comment seems completely out of place in a paper supposedly about natural testosterone levels (but read on).
  • The paper notes the "virilised phenotype" of "some female athletes." In plain English that means that they have physical characteristics found in stereotypes of men, and not in stereotypes of women. This sort of policing of women's bodies is ever-present in these discussions.
  • It acknowledges the Chand vs IAAF 2015 CAS decision as the motivation for the research, but does not acknowledge the quantitative conclusion of that ruling which indicated that the CAS decision was based on a supposition that T levels in women might account for a ~3% difference in performance but not a ~12% difference common to males vs. females.
  • The analysis looked at female and male athletes participating in the 2011 (female) and 2013 (female and male) IAAF World Championships. 
  • The study, oddly, includes independent results for athletes who participated in both 2011 and 2013 World Championships. It appears that these athletes were thus double-counted. The paper says that it is not an issue, so why do it at all?  It is inelegant at the least and problematic at worst.
  • The study focus on the athlete's single best performance in the competition, not overall performance. It would have been nice to see the sensitivity of the results to this methodological choice.The paper also aggregates all athletes' times into averages, another important methodological choice.
  • So, rather than present the data as a scatter plot (time/distance vs. T), which would allow a sense of variation in any possible relationship, the analysis used "tertiles" (thirds) and compared time/distance of the bottom third (in T) with that of the top third. It is an interesting methodological choice, as it all but eliminates the possibility to see and understand individual variation, e.g., in technical terms, least squares regression vs. Chi-Square test. 
  • The paper appears to include athletes who doped in the analysis of athletes with naturally high T. It thus mixes known doped athletes into the results, without quantifying the impact of this methodological choice. This is remarkable. The paper states:
    • "Among the 1332 female observations, 44 showed an fT concentration >29.4 pmol/L.17 Twenty-four female athletes showed a T concentration >3.08 nmol/L which has been calculated to represent the 99th percentile in a previous normative study in elite female athletes.13 Among these 24 individuals, nine were diagnosed with a condition of hyperandrogenic disorder of sex development (DSD), nine were later found to have been doping, and six athletes were impossible to classify."
  • The paper says that "In male elite athletes, no significant difference in performance was noted when comparing the lowest and the highest fT tertiles." This overall aggregation is not quite accurate. For instance, for the men's 5000m the lowest T third ran 822.96 seconds and the highest third ran 812.89, a difference of more than 10 seconds. Maybe high T men should be excluded from the 5000m? (I jest, but that is the logic at work here.)
  • The paper concludes, accurately, "Our study design cannot provide evidence for causality between androgen levels and athletic performance"-- this is both the nature of statistics, but also a consequence of the methodological issues this paper has.
  • Interestingly (and a side note to the focus of the paper), the paper notes that some of the observed low T numbers among male athletes could represent the results of previous doping, implying that these results are in some way contaminated by doping in a different way than the female results.
  • This is a remarkable admission: "we deliberately decided not to exclude performances achieved by females with biological hyperandrogenism and males with biological hypoandrogenism whatever the cause of their condition (oral contraceptives, polycystic ovaries syndrome, disorder of sex development, doping, overtraining)."  The Chand 2015 CAS ruling applies to women with high natural T, not doping or medical consequences (e.g., possible TUE). The study consequently mixes in some apples and oranges. This alone undercuts this study in the context of the Chand ruling.
  • The paper appears to address Caster Semenya directly when it states: "In female athletes, a high fT concentration appears to confer a 1.8–2.8% competitive advantage in long sprint and 800 m races." Interestingly, despite the paper's methodological issues, this is just about exactly the range postulated in the 2015 Chand CAS decision.
My bottom line: The paper has some significant methodological issues, most notably the inclusion of female athletes who doped with those with naturally high levels of T. There is some double counting of athletes in 2011 and 2013. There is also speculation that the male findings are contaminated by doping. Methodological issues notwithstanding, the paper nonetheless strongly reinforces the 2015 CAS Chand decision. There is nothing here that would provide any empirical basis for revisiting that decision. We might quibble about the methods, but the significance for the CAS decision seems unimpeachable.

Saturday, July 8, 2017

The Curious and Complex Case of Alex Schwazer

"Do they realise they are part of the plot against AS and the potential consequences for them?"

This statement, by an IAAF official to a to an attorney representing the IAAF refers to an athlete  - Alex Schwazer, AS -- who WADA and IAAF are collaborating together to convict of a doping offense. The "them" that is being referred to here is the WADA laboratory in Cologne. The "plot against AS"?  Well, that is a curious phrasing.

This statement can be found in a tranche of emails involving the Schwazer case released by Fancy Bears earlier this week.

Even before the leaked emails, the Schwazer case has been much discussed as problematic in many respects.  Here I'll list some resources for this case as a starting point for discussion.  The case is complex, involves many personalities, agendas and accusations. It also has a history going back many years, centering on Schwazer's coach, Sandro Donati.

I don't understand all the ins and outs, but I'd sure like to.

What else should I list here?

Monday, June 5, 2017

Guest Post: The "Arnie Effect" vs. The "Tiger Effect" on PGA Purses

This is a guest post by Bill Mallon, former PGA Tour professional and current surgeon and Olympic historian. Find him on Twitter @bambam1729.

Roger Pielke has written of the “Tiger Effect,” in this case defining it as the increase in purses on the PGA Tour after Tiger’s ascension to the top. Roger and I had emails back and forth on this as I said I thought the “Arnie Effect” may have been just as large in the 1960s, if not bigger.

Frank Beard, leading money winner in 1969, once said that all pro golfers should give Arnold Palmer 25 cents out of every dollar they earned, because he had gotten at least that much for them. A disclaimer here – in the 1970s, I was one of those pro golfers who owed Arnie a part of my winnings, playing from 1975-79, and if you want to know how much purses are different now than then, in 1977 I was 96th on the money list, with just under $24,000 official earnings. In 2016, the 96th player on the PGA Tour money list (Brett Stegmaier) earned $1,086,714. (Note: I am not bitter.)

So, I decided to look at the stats on this to see how much effect Tiger and Arnie had on PGA Tour purses. I chose to look specifically at the 10-year period starting when they first became the leading players on the tour. For Arnie that was 1958, while for Tiger it was 1997.

In the accompanying table [displayed below this post], you can see the PGA Tour total purses and how much they won going back to 1938, the first year that was recorded, as well as the number of events held. Because total purses are dependent on the number of events (take a look at the war year 1943 when there were only 3 events), the next column lists the average purse per event, which is a better statistic to use. Finally, to effectively use the same dollar values, this number is corrected for inflation, listed in the 5th column from the left.

The next column, P/E Adjusted, lists the Purse/Event Adjusted for inflation. This is the number we want to compare, but I went a bit further. Because there are some slight yearly deviations, I then created a 5-year rolling average of the Purse/Event Adjusted for inflation. Finally, the right-most column is what we are comparing – this is the 10-year increase in Rolling Average of the Purse/Event Adjusted for inflation.

The pertinent 10-year periods, the Tiger Era, and the Arnie Era, are marked in bold and highlighted in orange. You will note that during the Tiger Era, from 1997-2007, actual purses/event increased to 284.2%, which is quite good. In fact, the increase was even more than that in 2004-06, reaching 289.6% in 2005. In the Arnie Era, 1958-68, the actual purses/event increased even more, to 325.7%. Further, this increase continued into 1969-71, and topped out in 1970 at 355.3%.

Also of note, there is no other 10-year era that approaches the effect that Arnie and Tiger had on PGA Tour purses. The closest thing to it comes in 1992-95 when the 10-year actual purse increase was up to 218.2%. What could explain this? If anything, this should probably be called the Deane Effect, in honor of Deane Beman, then the PGA Tour Commissioner. In the late 80’s and early 90’s Deane started building Tournament Player courses, popularizing stadium courses, most notably at TPC Sawgrass, home of The Players Championship. He deserves a lot of credit for starting the increase in PGA Tour purses that has continued, led by Tiger’s popularity, into the 21st Century.

So my original suspicion, that Arnold Palmer affected purses even more than Tiger Woods did, was somewhat correct, although the differences between the two eras were not that large – maxes of 325.7% vs. 284.2%. They both had profound, and close to equal effects on PGA Tour purses, but Arnie remains “The King.”

Friday, May 26, 2017

Submission to European Athletics on Rewriting the Record Books

Submission to European Athletics on Word Record Proposals
Roger Pielke, Jr., Professor and Director
Sports Governance Center
University of Colorado Boulder
26 May 2017

European Athletics has proposed a new set of criteria for the validation of world records in the various disciplines of athletics (track and field). A world record will be recognized by European and world officials if and only if it meets the following three criteria:
  1. The performance is achieved at competitions on a list of approved international events where the highest standards of officiating and technical equipment can be guaranteed
  2. The athlete has been subject to an agreed number of doping control tests in the months leading up to the performance
  3. The doping control sample taken after the record is stored and available for re-testing for 10 years.
Because the IAAF began storing blood and urine samples in 2005, many have interpreted the new policy to mean that all records established before 2005 would be erased.

I wish to applaud European Athletics for initiating an important discussion about the credibility of world records in athletics. Here I offer a critique of the proposed new criteria. I argue that an evidence-based understanding of the problem being solved is needed before seeking to implement solutions. In this case, there exists a lack of evidence in support of the efficacy of proposed solutions, most notably evidence to suggest that athletics completions take place on a solid foundation of anti-doping regulation.

So what is the problem to be addressed, anyway?

European Athletics offers three “main reasons” why action is now needed (PDF):
  • To ensure that today’s generation of athletes are not chasing records set in completely different circumstances
  • To restore credibility to the European (and World) records list 
  • To regain public trust 
Let’s take a closer look at these reasons for action one at a time:

1. Difference circumstances

If there is one constant in the circumstances under which elite athletics take place, it is change. There is no stasis – in the rules that govern competition, in the technology used in sport or in the efforts to contravene and enforce the rules.

For instance, the IAAF recently approved new rules governing shoe technology, presumably in response to the new Nike shoes that allegedly provide Nike athletes of today an advantage that other athletes and all past athletes did not have.  The IAAF rulebook is a living document and athletes compete under the rulebook in place when they compete.

The intent of the European Athletics rules changes is no doubt to focus on doping in particular. The science of doping and anti-doping regulation is constantly changing. For instance, athletes who competed in 2005 did so under a Prohibited List with about half the substances on it as compared to athletes who compete in 2017.  WADA’s methodological guidelines for the detection of substances changes over time periods longer than a 10-year statute of limitations – Athletes who competed in 2005 are accountable under scientific detection methods available to 2015, whereas athletes who compete in 2017 are accountable under scientific detection methods available to 2027. The science of anti-doping regulation will advance between 2015 and 2027 (and so too will the science of doping and evading detection).

Elite athletes of different generations will compete under different circumstances. This is unavoidable. Drawing a line in 2005 – or 1990 or 2017 – is arbitrary. Sure, it can be done, but it would not address the concern of athletes competing in different circumstances.

The elephant in the room is a presupposition that records set prior to some arbitrary date are more “credible” than those set afterwards. This is a testable proposition, and the subject of the next section.

2. Restoring credibility

There is no doubt that some records of the past were achieved by athletes who broke rules that prohibited doping or, if rules were not broken, used the assistance of substances that were subsequently banned. Strong circumstantial evidence for this conclusion can be seen in the following graph created by the Financial Times.

If past records were achieved through the use of prohibited substances then the historical record of record-setting suggests that 1989 offers a clear point of demarcation for women’s events, but that no such clear date exists for men.

Another approach to establishing credibility would be to demonstrate with evidence that the prevalence of doping among elite athletes was significantly higher before some date, after which records could be considered less tainted than those which came before. 

Unfortunately, data that would provide evidence for trends in the prevalence of doping among elite athletes has not been collected.  Data that is available from rigorous studies suggest that doping prevalence among elite athletes in recent years in athletics could exceed more than 40% of all athletes (see de Hon et al. 2014 and Ulrich et al, in press).

In the absence of data on doping prevalence, efforts to establish “credibility” risk being seen more as window dressing and public relations, rather than evidence-based, rigorous and trustworthy. Credibility will best come from evidence, not exhortation.

3. Regain public trust

Similarly, it is not clear that public trust has been lost or even if was ever there in the first place. As one study concludes: “Despite the vast amount of literature available on doping in sports, little is known about how the general public actually thinks about doping.”

Further, it is not clear if public trust in the integrity of sport is affected more by evidence that athletes break rules or revelations of scandals among athletic administrators.  As suggested above with respect to doping prevalence, efforts to influence public opinion related to athletics records and, more generally, athletics integrity, will benefit from actual evidence of public opinion, what shapes it and why it matters for sponsors, fans and athletes.

Bottom Line

A decision to reset the record book related to athletics can be justified for any number of reasons. However, if the goal of such a clearing of the slate is to create more of a level playing field for verifying records over time, then the proposed approach by European Athletics remains premature, for reasons argued above.

An alternative approach would be to first address issues of integrity in sport by improving WADA and anti-doping regulation. Central to such improvement is to place anti-doping efforts on a more solid foundation of evidence and science.  In the absence of such improvements to anti-doping regulation, efforts to rewrite the record book will be undercut by the very first, inevitable scandal to occur in the coming years. Doping remains prevalent in sport, and creating a new record book won’t change that fact.

If and when anti-doping regulation is placed on a more solid foundation, then the time might be right to discuss a new era of performance and achievement.  Until then we should let past records stand as an indication that the work of anti-doping reform remains to fully be done.  

Monday, May 22, 2017

IAAF Changes its Shoe Rules (Again) and CAS Awaits a Case

As you might expect, the IAAF has rules in place governing shoes used in athletic competition. In The Edge I explain that one of the first rules for shoes was put in place in the 1950s after a clever Russian high jumper was launching himself off of platform shoes.

The rules governing shoes, and prosthetics used by Paralympians, are under sections 143 and 144 in the IAAF Rulebook.  These rules have changed a lot over the past decade, specifically in 2009, 2010-2011, 2012-2013, 2014-2015, 2016-2017 and now, 2017-2018.

The most recent changes to the rules can be seen in the figure below, screenshotted from the recently released amendments to the 2016-2017 IAAF competition rules.

The new rules are likely introduced in the context of controversy and discussion of a new Nike shoe, designed to provide runners additional assistance. I discussed some of the issues associated with the new shoe technology in The Guardian earlier this month. 

The newly adopted rules changes take a badly-written rule and make things worse.

For instance, the previous version of the rules explained that shoes "must not be constructed so as to give an athlete any unfair additional assistance, including by the incorporation of any technology which will give the wearer and unfair advantage." The phrase "additional assistance" refers to additionality over running barefoot. 

The new phrasing "must not be constructed so as to give an athletes any unfair additional assistance, including by the incorporation of any technology which will give the wearer any unfair or advantage."

It seems clear that the new language is crafted to eliminate controversy over the new Nike shoe because it removes the notion of "additional assistance"and the specific reference to shoe technology. 

What then is "unfair assistance or advantage"?  It is undefined.

As I argued in The Guardian we can look to Rule 144 governing prosthetics for insight to what "unfair" actually means in an IAAF context. There I explained:
In 2015 the IAAF quietly changed the requirement that it had to show an “advantage” provided by technology in order to ban an athlete. The rule change meant that the burden of proof was now on the athlete to show that the use of technology would “not provide him with an overall competitive advantage over an athlete not using such an aid” . . .

Thus, if we apply the same standards to Nike’s fancy new shoes that the IAAF applies to prosthetic limbs, then the shoes clearly are illegal under IAAF rules. They provide an overall competitive advantage over athletes not using the shoes. That is both what they were designed to do and also what is indicated by testing by my colleagues here at the University of Colorado. Not all athletes can use the shoes, because not all are sponsored by Nike. For the shoes to be allowed, proof would have to be provided that they do not provide an advantage.
As I concluded in that piece, thus IAAF has one set of standards for Olympians and Paralympians. They are inclusive for Olympians and exclusive for Paralympians. This would seem to be the dictionary definition of discrimination.

The new IAAF are a CAS case waiting to be heard. I expect that it will not be long before an excluded Paralympian takes this up. Watch this space.

Thursday, April 27, 2017

Twitter Talk: Scientific Integrity and Anti-Doping Regulation

I have just embarked on an experiment in communication. I have created a "TwitterTalk" of my presentation yesterday at the Norwegian Academy of Sciences and Letters. The talk is titled "Scientific Integrity and Anti-Doping Regulation."

You can see the whole thread starting here.

Comments welcomed as it is a paper in progress.

Tuesday, April 18, 2017

Oslo Forum on Doping with Independent Experts: 26 April

Doping: science, ethics and law

Location: Norwegian Academy of Science and Letters, Drammensveien 78, Oslo, Norway
26. April 2017 15:00

Sport has great social importance and popularity, requires large resources and receives much public attention. But the sport's values ​​are challenged by doping. In Norway, processes against Martin Jonsrud Sundby and Therese Johaug have triggered vigorous debates on the relevant issues. Why are these matters so important? Is there a good match between people's sense of justice and what is actually happening in anti-doping matters? What rules are applicable? How should anti-doping work be regulated? Good answers require informed debate based on ethical, scientific and legal expertise.

If you'd like to attend register here.

  • 15.00: Opening by the president of the Academy, Ole M. Sejersted
  • 15.05: Roger Pielke, jr., Univ of Colorado: Scientific Integrity and Anti-Doping Regulation
  • 15.35: Michele Verroken, Sporting Integrity, Ltd.: Does anti doping serve sports and athletes or its own interests?
  • 16:05 Sigmund Loland, Norwegian School of Sports: The Ethical Dilemmas of doping
  • 16.30: Coffee break
  • 17.00: Jens Evald, Universitu of Aarhus: Anti-Doping - The balance between efficiency and the rule of law
  • 17.25: Erik Boye, Oslo University Hospital: Scientific variability and fallibility
  • 17.45: Odd O. Aalen: Statistical aspects: How to evaluate the uncertainty of diagnostic tests
  • 18.00 Discussion & invited comments
  • 19.00 End
Roger Pielke, Jr. has been on the faculty of the University of Colorado since 2001. He is the director of the Sports Governance Center within at the Department of Athletics, having Previously directed the university's Center for Science and Technology Policy Research. Pielke is the author of The Honest Broker: Making Sense of Science in Policy and Politics and The Edge: The War Against Cheating and Corruption in the Cutthroat World of Elite Sports.

Michele Verroken is a qualified arbitrator, mediator and adjudicator, a former teacher and lecturer in sports science and physical education. She is the founding director of Sporting Integrity and the Director of Ethics and Anti-Doping at UK Sport, Michele created the UK's Drug Information Database, education programs The, Independent Doping Control Officer training and national anti-doping policy based on ISO-certified standards. Michele has significant experience in anti-doping programs The at national and international level.

Sigmund Loland is professor of sport philosophy and the Rector of the Norwegian School of Sport Sciences (2005-2013). He has published extensively within at sports ethics, the ethics of performance-enhancing technologies, epistemology of movement, and the history of ideas in sports. Dr. Loland ice forms President of the International Association of the Philosophy of Sport (2002-03) and the European College of Sport Science (2011-13), and he is member of the Ethics Board of the World Anti-doping Agency (WADA ) (2004-).

Jens Evald is professor of sports law; Head of Sports Law Research Unit, Institute of Law, Aarhus University; Member of the Board of the Institute of Sport (1998-2007); Chairman of the Dispute Resolution Committee, Danish Kayak & Canoe Federation (2000-presented); Vice Chairman Danish Sports Law Association (2001-2005); Chairman of the Board of Anti-Doping Denmark (2006-2012); Member the Political Commission, Danish Football Association (DBU) (2016-2017). He is author and co-author of more than a dozen books and numerous at articles. His work includes books and articles on private law issues, legal history, legal philosophy, biographies and sports law.

Erik Boye is retired professor and department head, Institute for Cancer Research in Oslo. With a background in experimental cell biology and biochemistry he has a long experience with Biochemical analytic techniques. Through the last five years he HAS BEEN Involved in Evaluating the quality of anti-doping analyzes.

Odd O. Aalen is professor of biostatistics in the Medical School at the University of Oslo. He has beenworking on statistical methodology Applied two medical research. He Also has an interest in the statistical aspects of diagnostic testing.