Tuesday, October 25, 2016

My October Daily Camera Column: College Athlete Compensation

With more and more money pouring its way into college sports, college athletes and their representatives have been pushing for a bigger slice of the growing pie. Earlier this month, the Supreme Court decided not to take on a case brought by former UCLA basketball player Ed O'Bannon over the rights to athletes' names, images and likenesses. O'Bannon, who played in the 1990s, first brought his case after seeing himself in a video game and realized he wasn't being compensated.

The Supreme Court could have used the O'Bannon case to make a landmark decision on college sports, but chose not to. So that means debate will continue, especially about big-time athletic programs with annual revenues over 100 million dollars. The debate isn't about whether or not athletes should be compensated — they already are — but how and how much.

I see a framework already emerging for answering these questions. It has three parts: stipends, prizes and intellectual property. Let's briefly look at each.
Read the whole thing here, and please feel free to come back and comment.

Here are some resources for additional reading if you'd like to go deeper:
There is also an enormous academic and legal literature on issues surrounding compensating college athletes. Here are two papers that I had on my course syllabus earlier this year:
And you might also have a look at this debate between Jay Bilas and Oliver Luck, two smart and thoughtful experts with different views on this topic:

Thursday, October 20, 2016

Your Input Requested: A New Sports Governance Professional Masters Degree at CU Boulder

At the University of Colorado Boulder and CU Athletics, we pride ourselves on our engagement with the broader community. As such, we’d like to solicit your thoughts on a cutting-edge Master’s degree program we have under development.

We are developing a highly exclusive, intellectually rigorous, 12-month professional Master’s degree, with the goal of preparing students for careers in sports governance. The proposed program aims to focus initially on four themes:

-International sports governance
-Paralympic sports governance

You can learn a bit more about our proposal by having a look at this one-pager in PDF.

If you have an interest in athletics performance, the rules, values, and institutions that govern sport, we invite your feedback. If you are interested in contributing to the development of the program, please use the comments here or email roger.pielke@colorado.edu with your thoughts on the questions below.
  1. If you were to consider enrolling in a professional master’s degree program, would you consider enrolling in a 1-year professional Master’s degree program in athletics governance? What factors are important to you in making such a decision?
  2. What sorts of skills, knowledge and networking opportunities would you expect from such a program?
  3. Where do you envision future employment with such skills, knowledge and opportunities provided by such a program?
  4. Are you a:
    • Current student
    • Professional athlete
    • College athlete
    • Current professional
    • Other
Thank you.


The Athletics Governance proposal team
Sports Governance Center
CU Athletics
University of Colorado Boulder

WADA Research Project on Problems with EPO Detection

At the same time that WADA was arguing for the sanctioning of Steven Colvert, based on the robustness of EPO testing, it had funded a group of researchers in New Zealand to work of problems associated with those same tests.

The project described above is listed on the WADA website (here in PDF) and was funded for the period 2013-2014 (according to a research report of the University of Otago Heath Sciences department here in PDF). Colvert was sanctioned in 2014.

Here is a key passage:
Reliable, sensitive and specific measurement of the hormone EPO + related analogues, used to illegally enhance sports performance, is a continuing problem for WADA. To date, immunoassay tests for EPO suffer sensitivity and specificity problems due to the multiple forms of EPO normally present in blood and also from newer synthetic short forms that can be used by competitive athletes. 
Sensitivity refers to false negatives (dopers who escape detection) and specificity refers to false positives (innocent athletes wrongly accused).

Uncertainty and Cherrypicking in the Colvert Doping Case

The figure above shows results from three identical tests (SAR-PAGE) applied to the same batch of urine, in the doping case of Steven Colvert.

Yesterday, I was able to deduce that the WAADs response to Lab Times used the middle panel to make an argument in support of his conviction in response to the recent Lab Times critique. However, the WADA experts who testified against Colvert did not use or refer to that middle panel in their testimony, but instead focused on the Screening Test which appears as the left panel above. The panel on the right is the SAR-PAGE result from the B sample, that is the second batch of urine from Colvert's drug test sample.

Here is what these panels show: On the right side of each panel is a sideways bell-shaped curve, which reflects the concentration of EPO in the sample. The blue line is a threshold value, above which indicates the presence of synthetic EPO, that is, doping. If the bell-shaped curve extends above that blue line, then it is an indicator of the subject having doped.

Much of the discussion in the disciplinary hearing on Colvert's case focused on the Screening Test. There is almost none of the bell-shaped curve above that blue line, which led a WADA expert to explain that one has to be an expert to be able to see it as evidence of doping.

In contrast, the WAADs response to Lab Times published yesterday focused instead on the middle panel, which indicates a more substantial amount of synthetic EPO above the blue line. Note that the SAR-PAGE results are a subset of the full scientific evidence of this case.

Us non-expects can take away a few key conclusions here:

  • Neither in the Colvert hearing nor in the WAADs response to Lab Times have WADA experts engaged with the totality, and thus contradictions, in the evidence. These contradictions are a key point made in the Lab Times article;
  • One can easily cherrypick selected results from the available evidence to support whichever case one wants to make with respect to Colvert: guilty or not guilty;
  • The incredibly large range of conclusions from the same test applied to the same urine shown in the figure above for the three tests says something important about what appear to be large uncertainties in EPO testing;
  • Colvert became "more guilty" in the testing as WADA experts were asked to confirm their original (and marginal) conclusions in the follow-up tests to the initial screening. This situation lends itself to a conflict of interest among experts being asked to show themselves to be right or wrong in their initial judgment.
In the context of uncertain and contradictory results, due process for an athlete should require a rather high standard of proof for a conviction to occur. In Colvert's case, the A Sample Screening Test was judged sufficient to reverse the burden of proof such that he had to prove himself innocent.

However, even today neither WADA experts among themselves nor independent experts are in agreement over what the test results indicate.

It is true that Colvert has not proven himself innocent. That is probably impossible, as the original samples have all been destroyed. At the same time, it seems clear that the evidence does not prove Colvert guilty either.

Wednesday, October 19, 2016

Views of Science in Colvert Doping Case

The figure above shows my attempt at mapping out the view on the science of Steven Colvert's doping case among three different perspectives:
  1. the WADA lab scientists who testified giantsinst Colvert;
  2. the independent researchers from Norway;t
  3. the WAADs scientists who have reinforced the Colvert judgement.
They don't agree.

Dodgy Image in WAADs Response?

UPDATE: Via Christiane Ayotte on Twitter I have learned that she is using the confirmation test and not the screening test. The difference between the two tests is a point of dispute. The confirmation test was not cited in the Colvert hearing, the screening test was, so it is a bit odd that Ayotte's defense ignores the science used in the hearing in favor of an alternative explanation.

The figure above shows the A sample result from Steven Colvert's original report (left panel, here in PDF). The one on the right comes from Christiane Ayotte's defense of WADA labs in that case published today here as Figure 4.

WADA Labs Now Admit Mistakes Were Made in Steven Colvert Doping Conviction

Writing for the WADA labs, Christiane Ayotte, President World Association of Anti-Doping Scientists, has authored a defense of the conviction of Steven Colvert for doping. But buried in her defense is an admission that should all but seal the case for revisiting, if not outright dismissing, his case.

Ayotte writes:
If the laboratory expert was correctly quoted [during the hearing], he made a mistake when he stated that the amount of recombinant was small when compared to the endogenous EPO.
Let's check the transcript, where not one but both WADA lab experts testified that the amount of recombinant EPO was small compared to endogenous EPO.

Phillip Reihlen of the WADA Cologne laboratory stated in his testimony (p. 66 in this PDF, emphasis added):
[Colvert's urine test] leads to a suspicious finding because we see a mixed profile containing mostly endogenous EPO with a small amount of recombinant EPO.
Christian Reichel or the WADA Seibersdorf laboratory stated of Colvert's test results (p. 111 in this PDF, emphasis added):
So what you see actually is this mixture of endogenous EPO and a small amount of recombinant EPO
In the Lab Times critique of the science underlying Colvert's conviction (PDF), the Norwegian scientists argued that the WADA labs presented inconsistent evidence indicating that Colvert had both a relatively small and a relatively large amount of synthetic EPO in his test results.

Ayotte confirms that when she says that her colleague (and actually, colleagues) "made a mistake" in their testimony.

On that basis alone, Colvert's case deserves to be reopened if not outright dismissed. An athlete's career is far too important to be allowed to be destroyed by such a poor and inconsistent application of science. That WADA scientists would defend each other is not surprising, however, at this point any fair-minded observer should be able to see that due process was not granted in the case of Steven Colvert.

Monday, October 17, 2016

Structural Biases in Anti-Doping Against Innocent Athletes Falsely Accused

The standards that WADA's Laboratories must follow are described in a January, 2015 document called "International Standards for Laboratories" (here in PDF). That document has some interesting regulations in it, that were I an athlete, would have me concerned about due process in anti-doping.

The first is a requirement that drug tests administered under the provisions of the WADA Code need only be stored for a minimum of 3 months. That includes positive tests - adverse analytical findings in WADA's jargon. This explains why the samples taken from Steven Colvert were destroyed long ago. Had they been saved, resolving his guilt or innocence with more advanced drug testing methods might be straightforward today.

There is an exception (
If the Laboratory has been informed by the Testing Authority that the analysis of a Sample is challenged, disputed or under longitudinal investigation, the Sample shall be stored frozen and all records pertaining to the Testing of that Sample shall be stored until completion of any challenge or investigation.
Any athlete, like Colvert, who believes that an adverse finding has occurred with respect to their urine or blood sample should -- as a matter of procedure -- immediately challenge or dispute the finding in such a manner that requires the sample to be preserved.

More generally, it should be a general policy that all samples associated with adverse analytical or atypical results is stored for a lengthy period, such as 10 years.

What would this involve?

All Olympic drug tests are stored for 10 years, in hopes of finding future evidence of guilt among dopers who initially escaped detection. At the Rio 2016 Summer Olympics this totaled about 5,000 tests, according to reports. Compare this with 3,866 adverse and atypical findings across all drug tests administered by WADA-accredited laboratories in 2014 (here in PDF). The scale would thus be similar to what is already done.

A second significant regulation in the lab standards is the reversal of burden of proof upon an adverse analytical finding. This means that upon a positive drug test, WADA "presumes" that the test is accurate and at that point the burden of proof is upon the athlete to prove that the test is flawed.

Here are the relevant passages from the WADA standards:
This is the equivalent, in a criminal setting, to asserting that you are assuming innocent until proven guilty until you are charged with a crime -- at that point, you are assumed guilty until proven innocent. The burden would then be upon you to prove your innocence.

Fortunately, most criminal justice systems do not work this way. However, anti-doping regulation does.

The combination of destroying evidence on which convictions are based after only a very short time and the reversal of the burden of proof dramatically increase the odds that athletes innocent of doping will become wrongly accused and sanctioned. Athletes should demand that both of these regulations be changed.

Anti-Doping Drug Testing Should be Evidence Based, not "Trust Us"

In anti-doping regulation, drug tests typically play a central role in the identification of athletes who have broken the rules by taking a banned performance-enhancing substance. However, as many of us will understand from experience, medical tests are not always 100% accurate - they have a failure rate. Anti-doping is no different, and we should not expect it to be.

Consequently, scientific integrity is crucial to any successful anti-doping regime. Fallibility is not a symptom of failure, but a reality to be accommodated - whether in a justice system, in a rocket launch or in a medical procedure. Science is thus central to making such systems work as we'd like them to.

A key element of scientific integrity in anti-doping is to rigorously understand, in quantifiable metrics, the rate of false positives and false negatives in drug testing.

  • A false negative test occurs when the sample (e.g., urine or blood) contains a banned substance, but is not detected. The technical term for this rate is sensitivity;
  • A false positive test occurs when the sample does not contain a banned substance, but the test shows its presence. The technical term for this rate is specificity.
False negatives and false positives in drug testing should be contrasted with getting away with doping or an improper conviction of an innocent athlete. Drug testing is usually an important part of any sanctioning process, but is not the same thing. Here are some examples:
  • Getting away with doping. Lance Armstrong famously passed hundreds of drug test, but we now know he was doping and got away with it for more than a decade;
  • Conviction of an innocent athlete. The case of Andrus Veerpalu, a Nordic skier from Estonia, is an example of an athlete accused of doping via a drug test, but whose appeal was upheld by CAS.
In both of those cases, justice ultimately prevailed. But such justice is not guaranteed. With these distinctions in mind, we can revisit issues raises in the cases of Erik Tysse and Steven Colvert, both of whom have very compelling cases of being wrongly sanctioned for doping.

In the context of the Tysse case, WADA laboratory scientists engaged in an exchange with independent scientists in the academic literature over the issue of false positives in anti-doping drug tests. That exchange is not reassuring.

In an excellent paper titled, "Can we trust anti-doping?" Waaler et al. (2011) offer a provocative thesis:
In the hunt for doped athletes, little account is taken of the risk of condemning an innocent athlete. The doping hunters appear to be more concerned with catching all the guilty than with considering the risk that some test results may be false positives.
The paper is worth reading in full, but in short, their concern is that WADA labs are not aware of the validity of the drug tests that they apply to athletes urine and blood samples. Validity refers to the quantification of the rates of false negatives and false positives in testing.

The authors conclude:
When it comes to the case in question, we must express concern: From the Adjudication Committee’s ruling it is clear that they are not certain of the facts of the case and that their decision is based on a perception that the laboratory has followed the WADA’s procedures. The chairman of the Adjudication Committee, Lars Erik Frisvold, said quite frankly during the verbal proceedings that much of the discussion that took place was over his head. Is it acceptable adherence to due process of law that the Adjudication Committee, lacking its own expertise, blindly accepts WADA’s procedures?

For doping tests, as for all other biological tests, knowledge of the validity of the test is fundamental. Validity is expressed through sensitivity and specificity. Whether sensitivity or specificity is the more important depends on the condition the test is intended to reveal. Both a"re important for doping testing. It is important to catch all cheats, but it is also important to avoid convicting innocent persons. Perhaps it is better to let ten doped athletes go free than to convict one who is innocent.

In this article we have concentrated on doping with CERA, and have had to conclude that the control system has drawn its conclusions without being able to document the validity of the test. It appears that it is assumed a priori that specificity is 100 per cent for a combination of A and B samples, which would exclude the possibility of false positive results. We have shown that lack of documentation of the validity of the test makes a conviction unreasonable and may result in a miscarriage of justice. The necessity of documenting the validity of tests applies to all doping tests – to ensure legal protection for athletes.
WADA lab scientist replied to the article in the same journal ("Doping analysis on solid ground," Hemmersbach et al. 2011), and focus on specificity, or the false positive rate, with respect to the drug test that was applied in the case of Erik Tysse (CERA):
Specificity (selectivity) is the most important validation parameter for not producing false positive results. In the period 2000 – 2007, when CERA was not yet on the market, more than 8,000 urine samples were tested at WADA laboratories in Rome, Paris and Oslo using the analytical method for detecting EPO. None of these samples had an isoelectric EPO profile that was similar to a positive CERA test, and this demonstrates the degree of specificity of the chosen method.
In other words, there has never been a false positive test. The accuracy is 100%.

Waaler and colleagues responded in a letter (translated from Norwegian via Google Translate):
Hemmersbach and colleagues write that they have infallibility in doping test work. Those responsible and involved conclude infallibility in [WADA laboratory] procedures, well not surprising. But still there is a lack an evidence of validity.
Østerud and Skotland (2012) offered another response to the WADA laboratory scientists, "Doping analyses are not on safe ground":
In their comments on the so-called Tysse affair, Hemmersbach and collaborators claim that doping analyses are on safe ground and free of error. This is a remarkable statement, in light of the fact that 94 independent experts, including 45 professors, have signed a declaration saying that data produced by the WADA laboratory in Rome fail to detect CERA in Tysse’s urine.
You can read more about the details of the Tysse case in The Edge. In addition to claiming infallibility, the WADA laboratory directors also threatened the journal which publishes the original critique of the use of science in the Tysse case.

None of this drama would be necessary if principles of scientific integrity were upheld in the WADA labs. A good step in that direction would be to require that the quantification of the validity of drug tests used in anti-doping are made available, including the details of the data and methods that underlie such conclusions. In this way the evidence base used to convict athletes would be open to scrutiny from independent experts and anti-doping regulation would be placed on a firmer foundation.

Were I an elite athlete, I'd be demanding that anti-doping regulations be evidence based. Right now, that base is on more of a "trust us" basis. Cases like those of Tysse and Colvert put that trust into jeopardy. It need not be so.

Thursday, October 13, 2016

The Curious Doping Case of Irish Sprinter Steven Colvert

In Newsweek I have an article about the doping conviction of Steven Covert, an elite Irish sprinter who specializes in the 200m. My article was motivated by a new paper just our in the journal Lab Times (PDF).

I'd encourage you to read both in full, but long story short: there are some significant questions raised about the substance and process of Colvert's doping conviction. Here is a report on the controversy in the Irish media.

This post is for anyone wanting to discuss the issues raised in Lab Times or my Newsweek article. Over the next weeks I will be following up here with a more in depth discussion of several elements of this case and its broader significance.

Wednesday, October 12, 2016

More on Expert Advice at USA Cycling: Should I Stay or Should I Go?

Last spring, USA Cycling terminated their relationship with their newly installed chair of their anti-doping advisory committee, Paul Dimeo of Stirling University in Scotland. For background and commentary see here and here.

A member of that committee, John Gleaves of California State University-Fullerton, has recently wrritten an essay explaining his decision to remain on that advisory committee after Dimeo was sacked. Here is an excerpt:
As much as I was disappointed, I also was conceey rned. It appeared that USA Cycling expected members of the advisory board would not offer opinions contradicting any of their official positions. If this were the case, it clearly threatened my academic freedom and I was not prepared to make this concession. I firmly believe that academics have an obligation to share what they discover, regardless of how receptive the political or cultural landscape finds the results. Making matters more complicated, I appreciated how choosing to stay on the advisory board would give the tacit signal that I ascribed to all of USA Cycling’s anti-doping positions, which I do not. I had agreed to advise them on the specific aims of the committee, I had not agreed to being a spokesperson defending all of their positions.

After returning from Greece, I had set up what I expected to be a difficult phone conversation with Jon Whiteman. I needed to know my academic freedoms remained intact and that there would be no censor over my writing or research. We agreed that as long as I offered such positions as a faculty member at California State University, Fullerton and not as a USA Cycling Anti-Doping Advisory Board member, I was free to write and speak in line with my research.

As for my disagreements with certain anti-doping policies, such as my opposition to cannabis being included on the banned substance list, the obvious point is that those policies are not even decided by USA Cycling but by the World Anti-Doping Agency. USA Cycling was not asking for my opinion on that issue and I was free to hold this perspective. I would only be asked to advise on the policies USA Cycling could control, such as anti-doping prevention efforts within amateur cycling, and I had no problem doing my best to advise them on such efforts.
Unfortunately, the agreement reached between Gleaves and USA Cycling doesn't make much sense.

Like it or not, it is a fact Gleaves is both a professor and a member of the USA Cycling anti-doping advisory board member. He can no more offer advice from one role or then other than I can offer advice as an American or as a professor. I am both. If a newspaper chooses to identify Gleaves as a member of the USA Cycling anti-doping advisory board, when writing about his advocacy of removing marijuana from the banned list, they would be completely accurate. He is and those are his views.

Experts offer advice and decision makers decide if and how it should taken. Strong organizations routinely solicit more and more diverse advice than they can use, because diverse views and conflicting knowledge can inform better decisions. They clearly explain that their advisors are advisors and do not represent the organization, as they are not employees or officers.  It remains unclear whether USA Cycling is up to the challenge of receiving such advice.

Gleaves has written a thoughtful essay, which is worth reading in full. It will be interesting to watch how the committee performs and hear further reports from the committee.

Lab Times vs. WADA Labs, Round 2

This week, the German medical journal Lab Times has published a remarkable article, by Nissen-Meyer and colleagues, about the doping conviction of Steven Colvert, an Irish sprinter. That paper argues that the Colvert conviction rests on a shaky foundation of science. I'll have more to say about the substance of the paper shortly.

Here I'll note that I've learned today that the WADA laboratories are preparing a response to that article. This will not be not the first interaction between Lab Times and the WADA labs. A somewhat bizarre earlier interaction is described in the excerpt below from a Guardian piece I did last summer, drawn from The Edge.

In that piece, I explained problems in WADA's use of science in the doping case of Erik Tysse, documented in several Lab Times articles by Nissen-Meyer and colleagues. I observed that the WADA lab directors were none too happy about it and responded not with argument, but with something else:
But the story gets stranger. The editors of Lab Times revealed in November 2015 that Peter van Eenoo, the president of a group representing the Wada-accredited laboratories, sent a letter on behalf of Wada laboratories to companies that had advertised with Lab Times. The letter, sent in October 2105, complained about the Nissen-Meyer article published in Lab Times: “Your company is one of the companies… which has supplied us with instruments for many years. As you are an important sponsor of ‘Lab Times,’ we want to inform you that we consider that the [Nissen-Meyer] article and behavior of ‘Lab Times’ reflects badly on your company.” One of the companies that received the letter shared it with the editors at Lab Times, who wasted no time in publishing an excerpt as part of an editorial commentary about van Eenoo and his colleagues’ tactics.

The Lab Times editors described the message that they thought the letter was carrying:
The malicious intent and undercover nature of this letter is a clear attack on press freedom. In addition, it may inflict serious damage on the business of our publishers. Van Eenoo stopped short of open threats, of course. But his heavily implied message is clear: The 35 accredited Wada labs will no longer do business with companies that advertise in Lab Times.
I asked van Eenoo about the letter and his motivation for writing it. He explained that Wada’s ethical code prohibited him and his colleagues from responding to the Lab Times articles in the scientific literature and that any such response would have called attention to the claims: “As a lab, we are bound by Wada’s ethical code and cannot respond in such cases. In these cases all the other party wants is as much publicity as possible. By responding to this, they would have gotten exactly what they want.”

Setting aside the issue of doping, in terms of scientific integrity, the behavior of the Wada lab directors is troubling on several levels. First, if true, the allegations related to the manipulation of images used as evidence by Wada’s Rome laboratory constitute research misconduct. Unfortunately, such allegations are not isolated. Since Wada was created, more than a dozen of its thirty-five accredited laboratories have been suspended or lost their accreditation due to producing “sub-optimal work.” The Rome lab’s activities in the Tysse case, if they happened, are clearly sub-optimal. The Wada lab in Rio, where the Olympics are about to begin, has just been reinstated after being suspended.

Perhaps more troubling is that, when faced with evidence assembled by a team of independent academics in a popular German journal, the Wada lab directors responded with what appears to be a thinly veiled threat to the companies that advertise in that journal. An independent investigation of the scientists’ allegations would have been a far better response.
All of this is documented in The Edge.

So, as a discussion on the case of Steven Colvert is motivated by the new Lab Times article, it will be interesting to see how the WADA labs respond to this latest independent evaluation of their work in a questionable case. Will they address the legitimate (and troubling) scientific issues? Or will they again threaten Lab Times with economic consequences?

Stay tuned.

Wednesday, October 5, 2016

Treating or Cheating? The TUE Question

From Steve and Joel at The Outer Line ...

Recent fallout over the hacking of the World Anti-Doping Agency’s athlete medical data files has been far-reaching.  And a new question is coming to light: if the TUE database can be hacked, can the process of obtaining a TUE be hacked as well?

Cycling, like many other sports overseen by the WADA codes, allows athletes to receive TUEs from their respective national anti-doping organizations, but only after rigorous medical testing and diagnosis confirmation.  But TUEs not only affect the elite strata of the sport; they are also becoming a major factor in U.S. amateur racing.  Several Masters racers have recently been issued a special type of certificate called a Recreational Competitor TUE, and now many amateur racers are asking if competition in those categories will really be fair in the future.

In a new article, Treating, or Cheating? The TUE Question, the folks at The Outer Line explore the current TUE controversy, from the perspective of this recently instituted Recreational Competitor TUE, and what it may imply in broader terms for the future of the sport.

Monday, October 3, 2016

THE EDGE - Book Launch Week!

This week marks the official, Colorado launch of The Edge. I'll be signing/speaking in Denver, Boulder and Colorado Springs. Thanks to my publisher Roaring Forties Press for hosting.

Here are the details:

Book Bar, Denver, Tuesday October 4th, 1-3pm
The Rayback, Boulder, Tuesday, October 4th, 5:30pm til
UCCS, Colorado Springs, Wednesday, October 5th, 6PM

Sunday, October 2, 2016

SKINS Investigation Final Report

SKINS has released our (Ross Tucker, @scienceofsport and my) final report looking into allegations of cheating by Robert Young in his summer 2016 attempt to break the Trans-America running record.

At the top of this post is our executive summary (click to embiggen).

Here are all the details:
I am happy to take any comments or questions here or on Twitter.