Learning About Science in Controversy Through Deflategate

I haven't paid much attention to the whole "Deflategate" controversy, mainly because I think it is silly. However, the issues related to whether or not scientific analyses can help to identify whether wrongdoing occurred offers a great example of the limits of science and the power of procedure, with great relevance to more weighty issues in sport, like doping.

Delategate refers to allegations that in the AFC Championship game earlier this year the New England Patriots illegally manipulated the pressure in their footballs (each team provides their own balls) in order to gain a competitive advantage. In the Patriots previous game, the Baltimore Ravens made similar complaints.

The issue has consumed an incredible amount of air time and the NFL has responded by suspending Tom Brady, the New England quarterback, who had his appeal heard earlier this week. Despite the froth around the issue there is a learning moment to be found here having to do with the role of science in resolving adversarial claims.

To get to the bottom of the allegations the NFL did what many organizations do when there is a controversy -- they formed a committee and commissioned a report. A committee of three led by Theodore "Ted" Wells duly produced a 243-page report which concluded that based on the evidence the Patriots "more likely than not" tampered with the footballs. The study was reported by the media as implicating Tom Brady and the Patriots.

Everything looks oh-so-clear when there is just one expert opinion available. Controversy attracts expertise of course and Deflategate is no different.

So earlier this month some interested scholars at the American Enterprise Institute, a conservative think tank, produced their own analysis of the Deflategate controversy and the Wells Report. They were not impressed (and said so in the New York Times). Faced with a critique of the Wells report, much of the media coverage swung 180 degrees and became very critical of the Wells report, in effect defending the Patriots against wrong-doing.

Faced with competing experts, as so often happens, the next step in the process is to start looking at motives, interests, and funding. Perhaps truth might be found there,rather in analyses themselves. The Boston Globe asked: why are these individuals at AEI, who normally comment on tax policy and such, looking into the Patriots and the NFL? They didn't find anything.

Just yesterday, Steve McIntyre, known for being a thorn in the side of climate scientists via audits of some of their work, published his own critique of the Wells report. Like AEI, McIntyre finds the Wells report to be fatally flawed.

So who do you believe? Wells? AEI ot McIntyre? Well, who do you want to believe? There is plenty of evidence and analysis to pick and choose from to make a claim that "science" is on your side. Spend 5 minutes on sports talk radio and that is exactly what you'll find.

It is worth noting that the correctness or incorrectness of the Wells report bears only a tangential relationship to whether or not the Patriots broke NFL the rules. It is perfectly plausible that the Wells report is deeply flawed and the Patriots still broke the rules. Similarly, it is possible that the Wells report is mostly correct (despite its imperfections) and the Patriots actually did nothing wrong.

Thus, Defategate offers a great analogy for the role of science in many public debates. We often transform the issue that we care about -- Did the Patriots break the rules? --  into a technical issue and then debate the technical issue rather than what we really care about.

A further lesson is that if our public debates do devolve into a technical dispute, then we can find ourselves in a situation where arguing technical details offers essentially no route to a resolution, as pretty much any position can find some support, somewhere, in science. One could imagine the science of deflated footballs leading to multiple academic papers, even conferences and a professional society where experts can share notes.

These issues have broader relevance. Consider that Lance Armstrong famously claimed to never have tested positive for doping. That is pretty much true. Of course, at the same time he is now known to have been one of the most prolific dopers in the history of sport.  With respect to doping, if we wish to make better decisions, then we have to structure our procedures to lead to more reliable scientific knowledge, relevant to the decisions that we want to make.

The case of Deflategate illustrates that to better enforce rules of the game of football we ought to pay more attention to establishing better procedures. This would mean not having to rely on science as an arbiter of controversy, or by structuring procedures in advance in such a way as to maximize the utility of science in decision making.

Ultimately, the NFL itself bears considerable responsibility for the Deflategate controversy. As McIntyre observes, "More professionalism on these [NFL rule] protocols would have been expected at a high school science fair." the same can be said for doping in sport, where no one really knows how many athletes actually dope or the effectiveness of anti-doping efforts.

If we want science to help resolve controversies rather than stoke them, we have to pay careful attention to rules and their implementation. It is not as fun, I know, to focus on process when you could be having a good fight between competing experts. But ultimately, effective processes can often help to avoid such a fight in the first place, and less drama all around.