How Many Elite Athletes Dope? A Lot Says a New Study

A new study has just been published by the journal Sports Medicine which argues that, currently, between 14-39% of elite athletes are doping - that is, taking prohibited performance enhancing drugs. The Dutch authors, led by Oliver de Hon, the manager for scientific affairs for the national anti-doping organization of  the Netherlands, conclude:

All doping-related discussions and decisions would be strengthened if this vital piece of information, i.e. scientifically reliable information on the prevalence of doping, becomes more readily available.

Current data suggest that 14–39 % of elite athletes are doping, but this figure needs further confirmation in different groups of athletes with varying levels and backgrounds. Doping prevalence can be expected to fluctuate substantially between different groups. However, the prevalence figure can be expected to be far higher than the average of 1–2 % of athletes who are caught with doping substances, or their metabolites, in their system. There are many efforts underway to close this gap, but this process is by no means complete.

Evaluations of the prevalence of doping use are not only interesting for sports fans and journalists. They are necessary for anti-doping professionals to enable true evaluation of the effectiveness of their policies. If the non-dopers are cheated by the dopers too often, and when doping tests are insufficient to control doping use in a meaningful manner, anti-doping efforts are doomed to fail. This is not a problem for the anti-doping professionals, but first and foremost for the athletes they have vowed to protect. Tools to evaluate the prevalence of doping use in sports are readily available; they only need to be used more often.

Their conclusions are very much the same as the ones that I argued in my recent piece in Nature. The research by de Hon et al. adds some considerable empirical heft to calls for better data and methods in quantifying the prevalence of doping. Also, as officials involved in sports governance their calls for better evaluation of anti-doping policies should carry some gravitas.

Of note, the authors find that between 19 and 56% of athletes admit to using a permitted performance enhancing drug (nicotine) and conclude:

If the entire doping test system is indeed unable to keep the use of prohibited substances at a lower level than a permitted substance, it adds to the idea that current anti-doping testing is far from effective in curbing doping. It is also disconcerting that calls for more clarity in this area that were made more than 25 years ago have not yet yielded much progress.

The paper is perhaps the most significant one yet published on the prevalence of doping and can be found at:

 de Hon, O., Kuipers, H., & van Bottenburg, M. (2015). Prevalence of doping use in elite sports: a review of numbers and methods. Sports Medicine, 45(1), 57-69.

Evidence suggests that lots of elite athletes dope, anti-doping policies aren't working, and anti-doping agencies aren't doing much about it.

On Narrative, Data and the ASU Curtain of Distraction

In today's New York Times, economist Justin Wolfers has a front page article on ASU's "Curtain of Distraction.," a feature of its raucous student section at home basketball games. Before diving into this post, let me say up front that I'm a fan of Wolfers research and follow him on Twitter, where he has a great presence. He has a lot of smart things to say on a lot of important subjects, including sports, where he has done some significant work. In this post I'm going to critique his NYT article, which is just not up to Wolfers' usual standards. It also offers a cautionary tale for would-be "data journalists."

The ASU Curtain of Distraction a make-shift stage that the student section uses to try to distract opposing the opposing team when it is attempting free throws. As the opposing player stands on the free throw line the curtain opens to reveal all sorts of eye-catching displays, from a guy in a Speedo to kissing unicorns (see above for an example). Wolfers argues that the Curtain of Distraction provides ASU with a quantifiable benefit because it leads the team's opponents to miss free throws.

It's a fun news story that combines what is best about college kids supporting their home team and the ideal of data journalism to provide insights behind the story. It's likely that combination of narrative and numbers that landed this article on page one of the NY Times. Unfortunately, like a lot of what passes for "data journalism," when you take a closer look, the story and the numbers don't actually fit together. There is a deeper lesson here about the power of narrative over the substance of data. Let me explain.

Wolfer's article is about causality, and specifically it's about detecting the impact of the Curtain of Distraction in a time series of data on free throws in games that ASU has played at home in Tempe, AZ. There are of course many ways to look for signals in data using statistical methods. As far as methodologically challenging signal detection, this one is not too complicated, because the data is good and the phenomena to be explained are very well observed. Even so, the data do not lend themselves to unique or conclusive claims of causality.

But Wolfers thinks that they do. He explains what he did:

A statistical analysis by The Upshot — with an assist from Nick Wan, who runs the True Brain blog, and from Jan Zilinsky — suggests that the Curtain really works.

It appears to give Arizona State an additional one­ to two­ point advantage per home game, beyond the normal home court advantage. The Curtain may even have played the pivotal role in the Sun Devils’ recent upset of their state rivals, the Arizona Wildcats.

The easiest way to see the effect is to compare visitors’ free­ throw shooting percentage before and after the Curtain’s 2013 introduction. In each of the three seasons from 2010­-11 to 2012­-13, visitors missed 28 to 32 percent of their free throws. Last season, the Curtain’s first, the rate at which visitors missed free throws rose sharply, to 40 percent.

In the first 14 home games of this season — when the Curtain and its surprises have continued to appear — visitors have missed 36 percent of all free throws. If you didn’t know better, you might suspect that the size of the hoop had gotten smaller.

Given the timing, and the fact that players are powerless to defend a free throw, it seems reasonable to attribute this sharp change to the student high jinks, rather than any change of players or strategy. In fact, the statistics largely rule out competing explanations.

As I'll show, the last sentence is completely wrong, and the conclusion of detection a strong signal from the Curtain of Distraction - an effect of 1 or 2 points a game - is just not supported by the data, despite the appeal of the narrative.

One mistake that Wolfers makes is to look at all free throws. The Curtain of Distraction (CoD) appears only on one side of the court, the student section, which ASU opponents face only in the second half of each game. So to look for a signal of the CoD we should look for its effects in second half free throws.

What do we see when we break down ASU opponents' free throws by half?

In the first half opponents shoot 66.7% when not facing the CoD. In the second half that drops to 61.1%.

Aha!! ... we might say, the effect is 5.6%. But as any student of causality knows, detecting a difference is not the same as accounting for that difference. So let's take a look at individual games to see if we might account for that difference.

It turns out that 8 of the 14 ASU opponents (including 7 of its last 9) actually improved their free throw percentages in the second half. Thus, there is very little basis in these 8 games to suggest that the CoD caused worse free throws. In fact, one could make a stronger case that the CoD improved second-half free throw shooting in the majority of ASU games. Of course, given any data, lots of plausible theories could be proposed. 

Looking game-by-game we can quickly see that through ASU's 14 home games this season so far, the reason that opponents shoot a lower free throw percentage in the second half can be entirely attributed to ASU's first two games of the season, Chicago St. and Bethune-Cookman. Chicago St. is ranked 333 of the 351 teams tracked by ESPN, and Bethune-Cookman is ranked 344. These are two of the worst teams in the country. One could plausibly come up with a large number of possible explanations for their poor free throw shooting in the second half - including the CoD - but also more prosaic reasons such as fatigue, or early-season road jitters, or the pressure of facing a much better team, or just chance, or something else.

Since those two games, in the subsequent 12 home games opposing teams at ASU have made 62.3% of their free throws in the first half and 65.0% in the second half. When facing the CoD teams have on average improved their free throw shooting! So even if we were to postulate a CoD effect in those first two games, it clearly wore off pretty quickly (and maybe even reversed;-).

Wolfers takes the causality argument even further when he writes that the CoD "may even have played the pivotal role in the Sun Devils’ recent upset of their state rivals, the Arizona Wildcats." Again, a wonderful story, even movie-script stuff, but the data just does not cooperate. Arizona was 0-2 from the line in the first half and 8 for 12 in the second half (66%). Arizona's season-long free-throw percentage is 69.2%. 

The bottom line here is that the evidence does not support, much at all and far from strongly or uniquely, a claim that the ASU Curtain of Distraction impacts opponents free throws in any quantifiable way. (For other analyses see the Harvard Sports Collective and True Brain). 

Sure one can spin a compelling narrative and find some numbers that seem to support that narrative. As the saying goes, numbers which are sufficiently tortured using statistical methods will ultimately confess. As a corollary I might add that pretty much any narrative one cares to spin can be supported by some plausible or plausible-sounding data. But I'm also pretty sure that is not how "data journalism" is supposed to work. 

As far as narratives go, the ASU Curtain of Distraction is a great one. Too bad the numbers don't follow along.

PS. If you'd like to play with the data from ASU's first 14 home games, here it is.

The (Mostly) Non-News of FIFA's Watered Down Governance Report

Earlier this week Der Spiegel reported that FIFA's "independent" reform committee had submitted a draft of its final report to FIFA, and made subsequent changes at FIFA's request. Someone apparently leaked internal FIFA emails.

As readers of this blog will appreciate, the news is really not news.

We have known for a while that the FIFA IGC (reform committee) was not independent and that its report was watered down (see this summary and full analysis in PDF). We also know that the FIFA IGC chair, Professor Mark Pieth, has faced a wide range of challenges dealing with FIFA, and has not always come away from the experience in the best light (just search this blog for "Pieth").

If there is little new here, the fact that this information has been leaked is itself interesting. Either the internal emails were leaked by Pieth or someone within inside FIFA's "football family." If the former then Pieth once again comes away the worse off, as FIFA's effusive defense of him (via this press release) simply provides more evidence of how conflicted his role of IGC chair actually was -- FIFA actually wrote: " People should stop questioning the independence of such a credible character." Ouch!

If the leaker is another FIFA insider, then it would speak to an internal split over the organization's leadership. However, were I to guess, I'd say that the leak most probably comes from Pieth. Any insider wanting to embarrass Sepp Blatter and also with access to confidential FIFA emails surely has to have better material than this weak tea.

Either way, the leak tells us nothing that we didn't already know: Sepp Blatter is well on his way to captaining FIFA to another term.

My Talk on "Sex Testing" Now Available

You can see my recent talk on "sex testing" in sport here. (Opens an AdobeConnect window)

If you'd like just the slides, they are here in PDF.

And if you'd like a draft for comment version of the full paper, just drop me an email (rpielkejr at gmail). I am happy to receive comments and appreciative of those who have already sent some.

Here is the abstract:

Sugar, Spice and Everything Nice:“Sex Testing” in International Athletics

Roger Pielke, Jr.
University of Colorado
Center for Science and Technology Policy Research

Abstract

In many settings, decision makers look to science as the basis for making decisions that are made difficult by their social or political context. Sport is no different. For more than a half century sports officials have looked to science to provide a clear distinction between men and women for purposes of determining who is eligible to participate in women’s athletic competitions. However, the science of sex provides overwhelming evidence that there is no such clear biological demarcation that differentiates men and women. Despite this evidence, the International Olympic Committee and the International Association of Athletics Federations in 2011 implemented a form of “sex testing” based on androgens, and specifically, testosterone levels in females. This paper evaluates this policy, finding it contradictory to scientific understandings of sex and counter to widely-held social norms about gender. The paper recommends an alternative approach to determining eligibility for participation in women’s sports events, one more consistent with the stated values of sports organizations, and more generally, with principles of human dignity.

New Resource: Human Diversity in Sport

Human Diversity in Sport is a new resource focused on increasing awareness of "human physiological diversity" in sport, with a focus on issues related to sex and gender. The resource has been created by Kristen Worley and Mianne Bagger.

The resource will be of particular interest to academics and others due to its searchable database of materials, including academic research, related to human diversity in sport.
They are also on Twitter at @HDiSport.