Starting this year, Major League Baseball (MLB) has instituted a new “automated ball-strike challenge system” (ABS), in which players can challenge the call made on a pitch by tapping their helmet. Shortly thereafter, on the stadium big screen, an animated illustration (familiar to fans of tennis, soccer, and cricket) of the baseball landing either inside or outside of the strike zone is used to announce an official determination of the call. Serious fans are aware this follows considerable technological changes (pitch clocks, replay review, etc.) that have been made in the last few years to improve the game and create ever more sophisticated data traces of what happened. And for these serious fans, much of the discussion about ABS has so far been on strategy — when should pitches be challenged during the game? — as well as on a newfound ability to determine how good umpires truly are (for which a convenient online tracking site has emerged).
For the rest of us, why might we care that a machine is helping umpires call balls and strikes? As with many technological “fixes” introduced into sports, the stated reason is to try to eliminate the possibility of an obviously bad call affecting the outcome of a game. Baseball has been relatively slow among major professional sports to adopt such technologies, and so perhaps it was overdue.
As a reason alone, though, it is lacking. Umpires, after all, were shockingly good at the task of determining the position of the ball when it crossed the plate, almost certainly the hardest task in all of professional umpiring and refereeing. The ball regularly comes at 95+ MPH, from a distance of less than 60 feet, making the ball’s total trip time from hand to glove something like 0.4 seconds. Add in movement in all directions, as well as the fact that the height of the strike zone is different for every player, it starts to become clear how hard this skill is to master (you can try it yourself online). The miracle, from this point of view, was that there were so few disputes over the thousands of these calls across a season.
But for those of us who follow baseball only casually, we should see this as a change of kind, not of degree, in the technological infiltration of the game. After all, even if it was supposedly well-defined, the strike zone has always been a zone that required interpretive judgment. Its origins in the 1850s were linked to what as “within fair reach of the batter,” as David Nemec’s 1994 Rules of Baseball pointed out. As I pointed out in my book Scouting and Scoring, this was consistent with many of the early decisions in baseball rules, such as the way fair and foul, or hits and errors were defined: with an emphasis on judgments of moral desert.
Such wide latitude did not last long, and soon “within fair reach” was redefined as “over the plate” and a height roughly between the batter’s knees and armpits. (In 1950, the top was defined as the armpits, but it was shifted to the shoulders in 1963, then back down in 1969, and adjusted to the “midpoint” between shoulders and the top of pants in the 1980s, and so on. The midpoint itself technically shifted depending on the way the batter stood at home plate, further complicating things.) Throughout, the formal definition was routinely ignored by umpires, and it was essential for teams to have a sense of what the umpire’s particular zone was on a given day. This created a sense of arbitrariness, but given the role of balls and strikes, arbitrariness was inescapable. Fairness — the original language of the zone — continued to be emphasized above all, both in terms of the contest between pitcher and batter, and in the relative calls for one team versus another. Nearly all fans grew up familiar with games that were called with generous zones (say, the second half of a double header, or a particularly hot day game, or one that featured a pitcher-favoring umpire).
In the current system, however, players are formally measured and the zone is defined as 27% to 53.5% of the player’s measured height, with an exact width of 17 inches. It might seem a simple formalization — the bottom of the knees redefined as 27% of height, but it’s not only a formalization. After all, the umpire now is making a decision — in real time — based on an incredibly precise measurement of height. If the zone had remained “bottom of the knees,” for example, a video replay might allow for some interpretive judgment about where the knees ended, allowing for egregious calls to be overturned and borderline ones to remain. Now, no such judgment is possible. This is perhaps a feature for computers, but a different expectation is put upon the human arbiters who remain: they are expected to anticipate the judgment of ABS. As the science studies scholar Harry Collins pointed out decades ago, when we make our intelligence and judgment more machinic, we make the possibility of artificial intelligence more plausible because we, as humans, voluntarily narrow the gap between what humans and machines do.
In Science and Technology Studies more broadly, the role of technology in sport has long been featured, both in terms of how it changes the way we see humans, and in terms of what we expect technology to do. For umpires, it is not clear how such a change will affect their role in the game — perhaps their authority will be strengthened because mistakes are easily rectifiable; perhaps it will be weakened as they are clearly inferior in judgment to that of the machine; perhaps it will simply result in a “different” view of them, judged not on their ability to ascertain where a pitch “was” but rather guess how the computer vision software places it vis-à-vis the invisible zone (Hessler wrote about this machine-human dynamic in the case of soccer). Whatever the case, umpires agreed to the use of ABS as part of their collective bargaining agreement and only time will tell how baseball fans and players’ view of the home plate umpire changes.
More directly, however, ABS marks a shift in what we expect of technology, as we’ve seen in nearly every other introduction of such devices into a sport. It is unlike the use of television replay, where the purpose is to “enhance perception.” When someone challenges a play on the field in baseball, video replay allows us the replay official as well as the rest of us to “see” whether or not there is “clear and convincing evidence” to overturn the call made on the field. With ABS, we are not shown the “raw” inputs into the automated system and we are not provided with such perceptual evidence as justification for the outcome. Rather, the video taken of the pitch from multiple cameras is analyzed in a proprietary fashion, and from that an animation is created which depicts “what happened” as a movie. The technology itself renders the decision on the basis of its own analysis of data. (This distinction between perception and decision rendering is made in a 2012 article by Collins and Evans).
It is not a trivial switch. For one thing, MLB decided to redefine the strike zone from a prism (the sides of which are defined by the edges of home plate and the height by the batter) to a rectangle situated in the middle of the plate. So, while the judgment was once about whether the ball crossed the plate at any point, now it is whether it touches a particular two-dimensional region. This was to prevent the machine from calculating a strike when a ball happened to slightly nick a corner of the plate at the front or back, even if for most of its trajectory the pitch was not a strike. But for nearly all of baseball’s history, that was entirely plausible (hence the phrase “must have caught the corner” of the plate, uttered sarcastically at a questionable call). Changing the formal definition of the strike zone in the name of avoiding “odd outcomes” and “producing results more in line with the expectations of players and fans” with ABS (https://www.mlb.com/interactive/mlb-abs-system-explainer) makes sense, but it raises the question about the underlying goal of engineering the system to match our human expectations rather than correct them when they’re mistaken.
This internal processing of the visual data raises all kinds of other questions. As is widely known, and as the manufacturer itself, Hawk-Eye Innovations, makes clear there is always a margin of error, both in measurements and in calculations. Officially, as reported by the New York Times, the league reports 95% confidence that a pitch is within 0.39” of the pinpointed location, and 99% confident it is within 0.48”. So, there is the first problem that the actual result does not indicate any error margins, even with a call is rendered on a decision smaller than a quarter inch (and well over 10% of pitches are in that margin, according to the same article). And those calculations don’t take into account that the very ball, itself, can vary in circumference of up to 0.25” (which, as school kids will know, when divided by pi, means the diameter can vary by nearly a 10th of an inch). These are incredibly small numbers, to be fair. The point is that these are the cases where challenges are most likely to occur given the rarity with which umpires make egregious mistakes.
The message of the system is an unchallengeable precision even when that is entirely unrealistic. There will never be uncertainty, even in cases that are, by definition, inescapably uncertain and within any plausible margin of error. One could imagine Hawk-Eye systems also using the “clear and convincing” evidentiary standard, interpreted to mean something more than 99% or 99.9% confidence in the result. But this third option (to let the umpire’s decision “stand” rather than confirming or overturning it) was rejected as implausible, in large part because once the automated rendering of the ball is shown, the expectation of perfect accuracy and precision renders a “we aren’t sure” decision into one that challenges the entire system.
Finally, it is worth at least pointing out that humans remain critical — they measure the players initially for the system and a “Ball-Out-Strike System (BOSS) operator” must put the right zone in for each player. Again, most of the time such human intervention will not matter, but it is telling that it is inescapable, no less than official scorers are still there making human judgments about what counts as a hit even if the result is seemingly objective data. Also telling is the system was implemented without simply having the home plate umpires receive a signal from the system as to what call to make. Such an implementation would mean there would be no human safety net if there was a glitch in the system. But there is also something reassuring about a human in “control” even if their decision-making ability is always subject, at least in theory, to challenges.
ABS seems unlikely to make controversy over calls go away. After all, what is the point of sport if not for uncertainty and capriciousness? It’s why we play the games instead of simply running computer simulations of them. The system will almost certainly reduce the disputes, in that it provides a “neutral” determination in which everyone can blame or thank the machine. But, there will be cases in which no challenges remain, or where the machine seems to fail or overturn an umpire’s decision in a case within the system’s margins of error.
We will see in the coming months and years how much this ultimately affects the game. On one level, it’s likely to be not all that much, given how good umpires were beforehand. On the other, we are now getting accustomed to outsourcing difficult judgment calls to machines in all parts of our life, often all too happily shoving the remaining uncertainty under the rug. It’s part of why AI is positioned to take over so many tasks: powerful forces convince us all of the fallibility of humans and the possibility of a more efficient technological future, without stopping to ask what this future will really accomplish or why we should want it. I, for one, will continue to lament the loss of the human element behind the plate in the name of possibly getting a few extra calls right. But it’s a price we are becoming all too accustomed to paying.
