IAAF on the Scientific Value of Blood Tests: 2011 vs. 2015

The IAAF, in a "response to allegations of blood doping in athletics" posted on Friday, alleges that the blood values of athletes obtained by a German documentary film maker have "no scientific value." Thus, the body argues that there is no case to be made against the IAAF for failing to act on what the data seem to show -- that there were hundreds of athletes with suspicious values whose cases were not properly followed up by the IAAF.

Specifically, the IAAF argues that very strict scientific protocols must be followed in order for the blood data to have any practical meaning, and these protocols were not introduced until 2009: "any apparent differences in results from one sample to the next have no scientific validity. Those standardised procedures were only introduced in 2009." Consequently, the conclusions that have been drawn based on the pre-2009 data are simply invalid.

However, the IAAF published a peer-reviewed paper in 2011 using the same or very similar data, where IAAF made a contradictory claim, that the blood data could in fact be used to estimated the prevalence of doping among elite athletes. The changing interpretation of validity of blood testing science by the IAAF is notable.

Here is what the IAAF said on November 27, 2015 (at 4.10, p. 14, emphasis added):

[A]ny competent expert would refuse to consider the values from the blood samples collected by the IAAF prior to 2009 as reliable evidence of blood doping, and would certainly have refused to compare the values from one sample in a profile to other samples in the same profile, because those samples were collected before WADA finalised and published the mandatory requirements for collection, transport and analysis of ABP samples that are set out in the WADA ABP Protocol, and so did not all comply with all of those requirements. This did not mean the values were worthless. To the contrary, they could be used to identify athletes who should be targeted for testing, in an effort to collect a urine sample with rEPO present in it, and when (as often happened) rEPO was found in the urine, that adverse finding would then be competent evidence of doping. But it did mean that the pre-2009 blood values could not be said to be reliable evidence of doping in and of themselves, for the reasons just explained.

Yet, in a 2011 peer-reviewed paper (Sottas et al. 2011 here in PDF) the IAAF said almost exactly the opposite. Specifically the IAAF noted that in blood testing, "the standardization was rigorous enough for drawing sound conclusions." That 2001 paper looks at the overall data to estimate the prevalence of blood doping among elite athletes, and found a rate of 18-19% among elite track and field endurance athletes, an estimate consistent with a 2011 study buried by WADA and recent research.

Here is an excerpt from that 2011 IAAF paper (emphasis added):

Let us present through a simple example how the prevalence of blood doping can be estimated thanks to hematological data. The hemoglobin of a population composed of undoped Caucasian male endurance elite athletes living at low altitude is well described by a normal distribution, with a mean of 146 g/L and an SD of 9 g/L. If a blood sample is collected from 200 of these athletes, between 1 and 9 of them (4 on average) should present a value higher than 164 g/L. If 30 of these athletes presented a value higher than 164 g/L, then between 21 (11%) and 29 (15%) presented a value that was too high. Only an external cause (doping or a medical condition) can explain this discrepancy. If the prevalence of the medical conditions is known to be low, then doping is the primary cause. Although this simplistic example is biased— superior methods that use or do not use population statistics of the relevant cause (here doping) have been described elsewhere (3, 10 )—it shows nevertheless that it is not necessary to have a test able to easily identify identify drug cheats to estimate the prevalence of test results attributable to the cause “doping”: A biomarker of doping with a known discriminative aptitude and the knowledge of the prevalence of confounding causes is enough. Today, the high standardization of the blood tests makes possible the estimation of the prevalence of blood doping by using epidemiological measures of occurrence.

The case being made by the IAAF in 2015 is apparently that the leaked (or perhaps stolen/hacked) data obtained by the German documentary film maker (ARD) is unreliable because it contains too many false negatives - that it athletes with seemingly suspicious blood values which have an entirely innocent explanation. The IAAF singles out Paula Radcliffe as an example of such a case.

While the case of Radcliffe is interesting, and worth a future post itself, as anyone familiar with statistics knows, arguing about a large data set by picking out specific cases is not a good route to robust interpretation of evidence.

Here is what IAAF can do to prove its case of many false positives: Apply the population-based statistical methodology that it used in Sollas et al. 2011 to the leaked data obtained by the ARD. If there are numerous "false positives" in the leaked data then the population-level analysis should show an incidence of doping that is statistically significantly higher than that reported in Sollas et al. 2011. This is just a logical point, easily addressed using mathematics.

If the population data is not distinguishable between the two datasets, then the IAAF has no case to make about the lack of robustness of the data set.

This is simple mathematics, and could easily be conducted by an independent party with all relevant identification metadata stripped from the data sets. In fact, to conduct the statistical analysis, which data set is which would not even have to be identified to the independent statistician.

Until IAAF resolves its internal inconsistencies on blood doping between 2011 and 2015, and applies simple and obvious statistical tests on the blood data, the organization will have a hard time convincing outside observers that it is worthy of any degree of trust in its claims.

Math can go a long way here. Use it.

TI Football Association Transparency Scorecard

Transparency International has compiled a simple scorecard of organizational transparency for the 209 FIFA member football associations. The overall results are shown above.

TI awarded one point for each of four categories:

We divided the information into four categories that represent basic information that should be available for organisations in order to monitor their governance and standards: financial accounts, codes of conduct, charters/statutes and information on activities. We gave one point for each of the four categories. The categories are derived from the Transparency International Business Principles for Countering Bribery.

Only 14 of the 209 FAs met these minimal standards (and, I note that US Soccer is not among the 14):

Only fourteen out of FIFA’s 209 football associations – Canada, Denmark, England, Hungary, Iceland, Italy, Japan, Latvia, New Zealand, Northern Ireland, Norway, Portugal, the Republic of Ireland and Sweden – publish the minimum amount of information necessary to let people know what they do, how they spend their money and what values they believe in.

Football's governance problems do not stop in Zurich or in the confederations.

Sunil Gulati's Answers Raise Many More Questions

Sunil Gulati has given his first meaningful interview in a long time to Sports Illustrated. He addresses several FIFA and CONCACAF issues. Gulati's answers in paces are misleading and in some cases, just incorrect. His statements lead to more questions about his role at the head of US Soccer and long-term roles in FIFA and CONCACAF. 

Here are some highlights that I have picked out.

Gulati is unconcerned about Sepp Blatter's revelations that the selection of Russia 2018 and USA 2022 was made in advance by members of FIFA's ExCo:

If the U.S. Soccer board or some subset of the U.S. Soccer board has an agreement about something, how they’re going to vote on an issue in two weeks, and three of their people change their mind, that’s fine. It’s fine for a subset of a board to have an agreement on whatever they want to do, unless that agreement is reached for inappropriate reasons. That would be different.

The first thing Mr. Blatter said was that we had an agreement to go to Russia because it was a big country and so on and so forth, and that’s what set off a lot of the issues in Europe … A board can decide and then change their mind—pretty simple. Unless the agreement was based on something inappropriate and that’s not what he suggested … three or four European voters changed their mind. They’ve got the right to do that.

I don’t attach too much to all that. That’s different than saying I don’t any concerns to the whole process.

Given the rampant corruption in the process documented by Heidi Blake and Jonathan Calvert in The Ugly Game, Gulati's reaction seems a bit blasé and incurious.

Gulati's apparent lack of concern about FIFA's awarding of the 2018 and 2022 World Cup tournaments are one reason why it would have been appropriate for him to have appeared before the US Congress, when he was recently invited. Gulati explains why he refused to participate and instead sent Dann Flynn, who put in a dismal performance:

It’s not as if we sent our summer intern. We sent the CEO of the organization and frankly didn’t know the other participants in the panel until a couple days before … Once you saw the other members of the panel then it was clear it was going to be more about [FIFA], but that wasn’t the focus early on.

Gulati is not playing it straight here. Before the Congressional hearing, the Committee's Chair, Senator Jerry Moran (R-KS), clearly explained the scope of the hearing well in advance:

The recent revelations of bribery and mismanagement at FIFA should be of concern to us all. The organization’s culture of corruption is turning a blind eye to significant human rights violations and the tragic loss of lives. This hearing on the recent FIFA scandals will begin the discussion about our country’s own participation in the organization, ways the United States and our allies can work to reform FIFA, and how we can restore integrity to the game so many Americans and citizens of the world enjoy.

Gulati also gives an unsatisfactory answer when asked about his relationship with Chuck Blazer and wrongdoing at CONCACAF, where Gulati was (and still is) on its Executive Committee:

For sure, Chuck and I were close for a long time. That’s true … Chuck was a good friend for a long time and a lifestyle that was different than mine, but was involved in financial markets separate and apart from what he did in the soccer world.

Gulati seems to be suggesting that his good friend's opulent lifestyle was funded by work that Blazer was doing outside of soccer.

You might want to believe that, until Gulati continues:

We had audited financials [at CONCACAF]. You rely on people that are in charge, especially when you have audited financials, that those are representing the organization. That’s completely separate from the things that were talked about in May [when indictments were handed down]. None of those would have ever gotten to the financials. So you’ve got two separate things. Think of the following: CONCACAF engaged in an integrity report and published results two or three years ago that looked a whole series of things. None of the things that came out in May were part of that report. 

Gulati's claim is simply remarkable, as it is clearly wrong. Here is what the CONCACAF Integrity Report actually concluded:

[Jack] Warner and [Chuck] Blazer, together and individually, used their official positions to promote their own self-interests, and frequently acted with disregard for the interests of the CONCACAF member associations and with disdain for the rules that governed their conduct. Furthermore, it is apparent that Warner and Blazer each was aware of the risk of potential misconduct posed by the other and was most capable of holding the other accountable; but neither did so, at least in part, to preserve the unfettered freedom to act in his own self-interest. 

The CONCACAF Integrity report alleges that some $88 million was misappropriated by Blazer and Warner. The report alleges explicitly that some of that money was used to  pay for Blazer's Trump Tower apartment, Miami South Beach apartments, Bahamas apartment, a Hummer and more.

Gulati says with a straight face that he believed that Blazer's lifestyle was supported by "financial markets" when CONCACAF had produced a report clearly showing otherwise, as Gulati sat on the CONCACAF ExCo? Seriously?

I really, really want to believe that US Soccer is among the good guys in the world of FIFA. I really do. But that stance is a hard one to adopt when Sunil Gulati is so obviously taking us for fools.

Op-Ed in the FT on the WADA IC Report

I have an op-ed in today's Financial Times on the WADA IC report. You can also read it below.

This week’s revelations of systematic corruption in the Russian athletics programme caps an annus horribilis for global sport. It is far from what was hoped for in 2007, when scores of nations agreed to co-ordinate their responses to doping.

The resulting treaty was probably the first in the history of the UN to claim both Palestine and Israel as signatories. The harmonised approach has helped to uncover, and to some degree root out, systematic doping in professional cycling — no mean feat.

Yet, despite such notable successes, the bodies that govern sport are ill prepared for the unfolding crisis in athletics.

An independent commission of the World Anti-Doping Agency has accused Russia of pressing athletes to participate in a doping programme, shielding them from detection by doctoring the results of anti-doping tests and then extorting money from them by threatening to blow the lid on their state-sponsored wrongdoing. The allegations are so serious that the commission recommended banning Russian athletics from the 2016 Olympics in Rio de Janeiro.

The International Olympic Committee asked the international body that oversees athletics competitions to “initiate disciplinary procedures against all athletes, coaches and officials who have participated in the Olympic Games and are accused of doping in the report”. But this body — the International Association of Athletics Federations — is itself among the accused (although most of the details have been withheld in the Wada report to allow law enforcement investigations to proceed). Lamine Diack, who stepped down in August as president of the IAAF after 16 years, is under investigation by French prosecutors on suspicion of taking more than €1m in bribes in exchange for covering up adverse doping results.

Efforts to police doping have focused almost exclusively on identifying and penalising individual athletes. The Wada code, which sets out the rules governing doping, has more than 5,300 words about how to punish athletes who transgress, but only 47 words on penalties for sports bodies and nothing at all on nations that break the rules.

But sport needs clean administrators and clean politicians just as much as it needs clean athletes.
Addressing the present crisis in sport will not be easy. But there are a few steps that make immediate sense.

First, the focus of anti-doping efforts needs to be broadened beyond athletes to develop mechanisms to hold sports bodies accountable.

The challenges here are not unlike those that have come to light recently in international football. But spotting that you need governance to improve is not the same thing as improving it in practice.

Second, the entire problem of doping in sport needs to be opened up to discussion and debate. Better evidence is needed on the actual performance-enhancing benefits and health risks of specific substances. Such evidence might be compared with the benefits and risks of performance enhancing substances that are legal, such as caffeine. Substances whose effects are “in the noise” might simply be ignored.

Simplification is needed because science and budgets cannot keep up with technological innovation in human enhancement. In 2004 there were 40 stimulants on the Wada list. Now there are 66 — and this is just a subset of prohibited substances. As a practical matter, a more realistic, focused approach is needed.

On present form, this year will probably be remembered as one of crisis in international sport. But it just might be remembered as the one that dragged sporting standards — and the sanctions for breaking them — into the 21st century.

WADA's Governance Challenge in One Graph

The WADA Code (here in PDF) overwhelming looks at the issue of doping in sport as a matter of individual athletes who violate the rules of competition. Hence, the WADA Code includes 5,329 words on sanctions by individuals.

The WADA IC report released today indicates that the IAAF and Russian doping violations were much more a matter of corruption by teams, sports bodies and nations. The WADA Code has very little to say on those issues.

The UNESCO treaty that legitimizes WADA and the NADOs offers little assistance. The excerpt below comes from this recent report from the October 2015 conference of parties to the International Convention Against Doping in Sport (here in PDF).

Sports governance is really into uncharted territory.

Video of Play the Game Debate on FIFA

Sepp Blatter's Salary Revealed, and My Estimate Was Spot On

On French language TV5 Monde, FC Sion president Christian Constantin reveals that Sepp Blatter's FIFA salary is "nearly 8 million Euros" and that of FIFA's secretary general is nearly 5 million euros.

I am going to give myself a high-five for my estimates, which appear to have been just about on target.

In 2013 I guesstimated ~$6 million per year in ~2010.
In 2015, I provided a range of $6 to $16 million, with $11 million as the midpoint.

Pretty close!

My New Office

FIFA Debate at Play the Game

I had the opportunity to moderate a debate at Play the Game last week which had two candidates for FIFA president, David Nakhid and Jerome Champagne (from right to left above starting with me in the upper right background). Also in the photo are Danish FA president Jesper Møller and Jens Weinreich, award-winning investigative journalist and author. (Copyright Thomas Søndergaard/Play The Game).

Since the debate, Nakhid has been declared ineligible by FIFA, has seen the Mail on Sunday allege a link to payments from Mohammed bin Hammam and stated that he is taking FIFA to CAS. Never a dull moment with FIFA.

You can read about the full debate here.

Sepp Blatter and the Richest 3,000 Swiss

In a "Lunch with The FT" FIFA president Sepp Blatter was asked if he was a rich man. His reply:

No, he says, he only earns what Fifa pays him — a sum that he refuses to disclose because Fifa releases its leadership payments only in aggregate; last year it paid $39.7m to its “key management personnel”.

“What do I do with my money? My daughter has an apartment. I have an apartment, one here and one there. That is all. I am not spending money just to show I have money. If you look at the richest Swiss people, I cannot approach the richest 3,000, because they are up to $25m.”

Setting aside whether $25 million might be classified as "rich," Blatter's numbers are a bit off.

According to the 2015 Credit Suisse Global Wealth Report (here in PDF at p. 112), in 2015 there are 3,737 Swiss individuals with a net worth of more than $50 million.

Earlier this year I estimated Blatter's FIFA salary at somewhere between $6 and $16 million dollars, basic on an analysis of FIFA accounting.

This salary estimate coupled with the fact that Blatter, when asked about his wealth, can quickly cite (dated) statistics on the Swiss richest 3,000, leads me to conclude that Blatter is indeed a very rich man.

$25 million? Take the over. You can bet the Swiss apartment on that. Both of them.