Umpires had an outsized impact on the Cubs series with the Phillies

I want to be really clear about what this post isn’t. This isn’t an argument that the Cubs would have won Thursday’s game if the umpires had called it differently. The Cubs had a lot of opportunities to score that they didn’t capitalize on, Jon Lester got hit hard early and the defense wasn’t exactly what we’ve come to expect from the Cubs. In fact, like in most games, while the strike zone was an issue it was just one, in a sea of many, many other issues. Incidentally, that’s basically the reason that all baseball fans are taught early and often not to blame the umpires. And I’m pretty sure it’s why my dad always said “Sara, both teams have to deal with the same zone. “

That said, it’s high time baseball do something about the capricious nature of the way balls and strikes are called. The Phillies and Cubs are both in first place in their divisions. They just finished a four-game series and this is how Andrew McCutchen drove in the Phillies’ ninth run on Thursday:

MLB Gameday

I can hear the dissenters now: “But, Sara, the Cubs only scored seven runs. That ninth run wasn’t the ballgame.” While reasonable people can disagree about plate approaches and strategies that can be deployed down one vs. two in a final inning, to quell those replies I give you the Gameday for the first batter Rowan Wick faced in the fifth inning, courtesy of a Twitter reply to me from IWWWallace:

MLB Gameday via @IWWWallce

I don’t want to get too technical about this, but that isn’t a 2-1 count, it’s a strikeout. It also clearly changes the way pitchers have to approach batters and the outcomes of at bats. Rhys Hoskins went on to single in that at bat and then came home to score on an Odubel Herrera ground out.

Honestly, if this were one aberrant game, I’d let it go, but it isn’t. This is the same series that saw this happen Monday night:

The umpiring in this series was atrocious, and baseball deserves better. Balls and strikes aren’t NFL catches. Fans of the game know what these look like, we literally all have a little box that tells us exactly where the ball is when it crosses the plate.

I guess I should caveat that “we literally all” comment, because a crucial demographic does not, in fact, have real time information about the location of pitches: the umpires in charge of calling balls and strikes.

Recent research suggests it is high time we change that.

As the 2019 season started a team of researchers at Boston University led by Mark T. Williams published a study that tracked over four million pitches thrown between 2008 and 2018. For the data nerds among you a caveat about their methodology before we jump into the findings. In that 11-year period there were multiple different tracking systems for balls and strikes. So the data from 2008 isn’t perfectly comparable to the data in 2018. That said, the triangulation systems deployed in MLB parks over that time period are all considered highly accurate. The Next Web, a scientific web magazine summed it up nicely:

In the 2000’s, MLB started to retrofit all 30 professional stadiums with triangulated pitch cameras. Each camera can track a ball up to 50 times in the period after it leaves the pitcher’s hand, and the time it crosses home plate. The cameras are said to be accurate to within an inch. Now we see this overlaid pitch data on every MLB broadcast.

This study has a few key findings and I’m going to try to summarize each one, but you all saw the main takeaway with your own eyes this week at Wrigley Field. Umpires are not good at getting close calls right, some umpires are substantially worse than others in predictable ways, and it absolutely changes the outcomes of games.

Umpires are not good at getting close calls right

If you read my comments in game threads or follow me on Twitter you know I’m an sort of addicted to screenshots. Crazy plays, fan reactions, oh, and blown strike calls. It’s a bit of a pet peeve, but like most fans I assumed for years that this was confirmation bias, me wanting my team to win. Me failing to account for the blown calls the other way. Well, it turns out umpires are not great at close calls and they have two key biases.

First, two strike biased called strikes. The Boston University team found that if a batter has two strikes on them the odds of a strike three call increase, even on a clear ball. While that number has improved since 2008, it’s still unacceptable with over 20 percent of the calls being blown:

Mark T. Williams, Boston University study 2019

But there is more, and it may explain why Kyle Schwarber is always shaking his head when he gets rung up for taking balls out of the zone that get called strikes. It turns out some blind spots are more predictable than others as you can see below:

Mark T. Williams, Boston University study 2019

Umpires and error rates are predictable

Additionally, the way umpires are currently placed does not lead to the most accurate results. I understand why the umpire union doesn’t want this information widely available, but at this point the data sets are public and I’m just conveying the reporting of a study. I think it’s fair to say that the ship has sailed on the idea of protecting some umpires over others. The Boston University Study was able to identify the most/least accurate umpires and it turns out that the system that MLB uses to assign home plate umpires to rotating four-man crews is flat-out bad for accuracy.

Here is their summary of the findings, for reference, BCR stands for “bad call ratio” which was generated by dividing the number of incorrect calls by the number of total pitches seen:

Based on the research, professional umpires, similar to professional baseball players, have a standard peak. The study revealed that home plate umpires who made the Top 10 MLB performance list (2008-2018) had an average of 2.7 years of experience, and averaged 33 years of age with a BCR of 8.94 percent. None of these top performers had more than five years of experience or were older than 37.

They continued:

Taking into account standard peaking, MLB should consider moving away from the traditional four-person crew rotation, which gives every umpire time behind the plate, no matter how young or old, experienced or not, or how strong or weak a performer they are. A better system would assign the top performers to the most physically and mentally demanding field positions. At some point, prime is reached, and surpassed, and the body and statistics do not lie.

To get a better idea of a good or bad BCR, here are the tables from 2018 that identify the top ten most/least accurate home plate MLB umpires:

Mark T. Williams Boston University Study 2019

Mark T. Williams Boston University Study 2019

This data suggests that home plate umpires should be assigned on merit as opposed to the traditional rotations we’ve seen in the past. For what it’s worth, none of the above umpires were on the crew calling this week’s Cubs-Phillies series.

Takeaways

I really want to give credit where it’s due. There are obvious improvements that are noticeable in the data. You can see it in the blown two-strike call table, but you can also see it below:

Mark T. Williams Boston University Study 2019

That said, is a 9.21 percent error rate good enough? Last year the Cubs missed a National League Division Series appearance on one game. In this four game series the umpires have made judgment calls that radically changed the outcomes of two games out of four. Is anyone reading this comfortable enough with home plate umpiring to say that it’s accurate within a one game margin of error? I am not.

The best umpire in MLB in 2018 had a Bad Call Ratio of 7.28 percent. Even if robot ump technology isn’t perfect yet, I imagine it’s got a lower error rate than that.

Bring on our robot overlords. Do it now.