tag:blogger.com,1999:blog-2807806767960745481.post3812504691823946187..comments2024-08-05T03:31:49.169-07:00Comments on The Least Thing: Bias and Accountability in the Context of Evaluation: Pitchers, Umpires and RaceRoger Pielke, Jr.http://www.blogger.com/profile/04711007512915460627noreply@blogger.comBlogger2125tag:blogger.com,1999:blog-2807806767960745481.post-68850959065298600072011-07-10T13:15:59.360-07:002011-07-10T13:15:59.360-07:00So the bias is less than a pitch per game? Could t...So the bias is less than a pitch per game? Could this measured bias be due to the very large sample size? Just as small samples sizes cause problems in statistics, large samples can as well. In large sample sizes, you can find statistical significance where there is no practical significance. That is, the 'bias' may be a product of the analysis and not of the umpire's judgement. <br /><br />A related problem is that in any 'finding racial bias' study, we can make a reasonable assumption that the authors of the study went into it looking for bias. When is the last time you saw a news article "Study Shows No Racial Bias in xxx?" Once you acknowledge that researchers go into a study looking for a particular result, it is reasonable to question the nature of the analysis. Did they use one statistical tool, find no bias, and shift to another method? There is a reason why the Analysis section of a paper is written AFTER the work is completed. An honest way of doing it would be to require a Methods and Analysis section to be submitted before the work was started. That would prevent data mining and analysis shopping for significant results. <br /><br />Of course, the 'less than one pitch per game' result has further ramifications. One pitcher rarely pitches an entire game. Relievers often pitch a single inning each, meaning that it would take more than nine games for the claimed racial bias to show up once. If we were talking about racial bias, and not a numerical bias of outcomes, wouldn't you expect it to show up by appearance, rather than by pitch? Black baseball pitchers are black all the time, not just once every nine-ten games. <br /><br />This factor and the large number issue mentioned above both point to the overriding issue of statistics - there's a difference between statistical significance and subject significance. When statistical significance has no practical effect on your subject, it should be ignored. All too often, in academic publishing, this prime rule is often ignored.Mark B.https://www.blogger.com/profile/03524735496130204611noreply@blogger.comtag:blogger.com,1999:blog-2807806767960745481.post-35655865351022741442011-07-07T23:55:40.502-07:002011-07-07T23:55:40.502-07:00Roger did the authors also take a look at whether ...Roger did the authors also take a look at whether the pitcher that was getting the favorable calls also has a reputation of very good control? A good example of this from the 1990's was Mike Maddox of the Atlanta Braves. During telecasts they would show that the home plate ump was more likely to call a close pitch in Maddox's favor over a pitcher that isn't considered to have good control. The pitch was in the same location but Maddox got it called a strike where his opponent's pitch was called a ball.<br /><br />Another thing to look at is that if one team is pounding another and it gets late in the game, the Umps will widen the strikezone against the winning team to get the game moving along.<br /><br />These points come from the perspective of someone that was a pitcher up through High School and whose coach all through Little league and up was a former Major League pitcher and played with and against former minor league players for years in the Navy. They are also well known "biases" among baseball players.<br /><br />As to finding that any "bias" disappears when things are at the end and on the line is not surprising since MLB has pounded umps to make sure they don't decide a game the players do. it's kinda similar to how in soccer you very rarely see a ref call a PK in the last minute of a 0-0 game.Anonymousnoreply@blogger.com