Analytics. Advanced Statistics. Enhanced Stats. Fancy Stats. The terms used to describe rapidly the evolving set of mathematical parameters by which the on-ice game can be judged vary widely, with some being perhaps a bit more derogatory then others. Whatever the terminology, the phenomenon has rapidly progressed from the passion of a fringe group to widespread acceptance. NHL clubs are increasingly incorporating analytics into their operations and evaluations, lending further credence to their long-term viability.
However, as with any new tool, there is an almost irresistible human compulsion to expand its utility well beyond what it was designed to do. Hockey analytics are no exception to this tendency, as Corsi/SAT, Fenwick/USAT and their derivatives are now being used as support for a wide array of propositions, some of which are of dubious validity. We’ll examine some of the practical and logical limitations of the advanced analytics — purely at a high level and from a logical/observational point of view. ( I’m targeting the off-season for a more detailed look at the use of analytics.)
Let me start by making it really clear that I am a big fan of properly used hockey analytics. I ‘m pleased that all of those boys and girls that sat in the front row of 8:00 AM Advanced Calculus, eagerly raising their hands, — while we History majors headed to law school slumbered in back — have a sports-related outlet for their talents. In all seriousness, the mathematics underlying analytics is fascinating, and well beyond my mathematical prowess. The concept is not a new one in our family, either. My wife — raised as a rabid baseball fan — is a lawyer who holds a mathematics degree and is a former member of the Society for American Baseball Research (SABR). We attended a SABR conference years ago, and I remember sitting glassy-eyed in one presentation from a guy straight out of The Big Bang Theory concerning a new proposed statistic. So, any effort on my part to undermine the inherent value of analytics could pose significant domestic consequences . . . I seriously welcome any cogent effort that would enable us to bury the plus/minus statistic once and for all, as truly one of the most awful, meaningless statistics in sports.
We’ll use Corsi/SAT as the basis for the discussion, as it has seemingly emerged as the pre-eminent tool from which the largest subset of more situational numbers are derived. For the uninitiated, Corsi/SAT is simply the shots attempted — whether on goal, blocked or missed. It can be aggregated on a team basis, evaluated overall, at even strength, and under a wide variety of game-specific situations. It is premised upon the assumption that shots attempted are a reasonable proxy for possession in the offensive zone. The theory here is that the longer the puck is possessed, the more shots will be taken. With this background, it seems apparent why Corsi/SAT has emerged as the leader over Fenwick/USAT (which excludes blocked shots), as it takes no less possession time to fire a shot that is blocked by a skater than it does to fire a shot that misses or is blocked by the goalie.
So, it’s not hard to see that Corsi/SAT basically represents a refinement (and improvement) over the plus/minus construct, which relies solely on goals scored at even strength. Put me out there wobbling along next to Wayne Gretzky, with a solid goaltender behind me, and I could rack up some big plus/minus numbers. Take a look at the plus/minus leaders every year, and you’ll find that they are dominated by players from the teams with the best records, and often dominated by fourth line players who are on the ice situationally. Corsi/SAT, on the other hand, attempts to more granularly quantify the element of possession, which should roughly translate to success.
I say should advisedly, because the correlation between Corsi and results is not perfect. Teams that focus on the transition game may not have great Corsi/SAT numbers, but will win a lot of games. Teams that gain early leads, then work to control the puck and limit the opposition to blue line bombs can lose the Corsi battle, but win the game. To analogize, how many times have you seen an NFL quarterback rack up huge passing yards in a losing effort? Similar concept here. Blue Jackets fans need only look to the last game against Colorado to show how there can be a disconnect between the analytics and the results. In that game, Overall, the Blue Jackets won the Corsi/SAT battle, 68% to 32%. At even strength, the disparity was even more dramatic, as Colorado managed only 19 even strength shots on goal, compared to 33 for Columbus. Yet, at the end of the night, the scoreboard read Colorado 4 Columbus 0. A hot or cold goaltender can alter the equation dramatically. However, applying the law of large numbers, upon which casinos and insurance companies thrive, we can perhaps say that these situations balance out over time. Still, it cautions against a religious adherence to Corsi/SAT as a predictor of success.
At the team level — and even at the unit level (i.e. combinations of three forwards and two defensemen), the Corsi/SAT number is a convenient and defensible tool to use as a comparison metric, at least on a relative basis. However, what I see with increasing frequency is the tendency to take Corsi/SAT (or Fenwick/USAT) and apply it to individual players and derive an absolute ranking of that player’s ability or value. It is here that the process begins to break down. (While there are other advanced stats, such as PDO and the like, which attempt to more specifically address individual performance,my concern here is only with attempts to extend the use of the basic advanced stats beyond their truly applicable scope.)
Any time you take a statistic designed to describe team play and apply it to individuals, there are problems. Again, if you happen to share the ice with Gretzky, you’re going to be in luck. Get stuck on a line with some bangers, or be on a team with substandard goaltending, the numbers will look different, all for reasons having only marginal connection to the ability of the player being evaluated. Sure, some effort is made to account for this by utilizing a “quality of opposition” metric, but that “quality” is judged by the same standards, so it becomes a bit of the dog chasing his own tail. If the metrics used for evaluating a player are suspect, and the opposition is gauged by the same metric, then the validity of the ensuing comparison is increasingly suspect.
Obviously, when an individual’s statistical representation is dependent upon what somebody else does, there is a lot of “noise” introduced into the analysis. Evaluating a goaltender or pitcher by wins is less telling than GAA or ERA, for example. Neither is perfect, but the latter is much closer to a representation of individual skill-based qualities. Walk outside at night and look at the stars. Sirius is the brightest star, correct? Well, not really. It appears to be the brightest due in large part to its proximity to us. Move all of the stars to the same distance, and it becomes a relatively ordinary star, but significantly brighter than the Sun, which would be barely visible to the naked eye. The point is that we are compelled to consider both perspective and circumstance when evaluating players, and the use of even advanced analytics to do that is an effort fraught with peril.
Consider a player on a team that thrives on the neutral zone trap. They win a lot of 2 -1 games and generate relatively few shots. Coaches can –and often do — elect to use players in particular situations, which may not necessarily correspond to their highest and best use, but is what the team requires at the time. Take a player from Team A and put him on Team B, and entirely different results can emerge, depending upon how he is used. For a local example, look no further than Nick Foligno. Would Joe Montana have been a Hall of Famer if he had been drafted by Al Davis and the Raiders, and forced to play in the “long ball” system? Perhaps not, but that says less about the inherent talent of the player than it does about the efficacy of the system.
While not a great tool for assessing absolute individual value, stats such as Corsi/SAT can have some value on a relative basis within the same team, where many of the variables are removed, and the players operate within a grossly similar environment. That’s a big component of their actual use today.
Like so many things — guns, the internet, alcohol — advanced statistics have a lot of intrinsic value and are useful in many circumstances. However, they can also be abused and misused, which is when their utility vanishes. As with any tool, enhanced statistics are merely a part of the complex matrix for evaluating a a game that inherently involves a lot of randomness, chance and variability. So, as we climb on the analytics bandwagon, let’s keep in mind what they do well, and what they don’t.
Editor’s note: A previous version of this article included a line that was offensive to women. We apologize. It was completely unacceptable and won’t happen again.
Author’s Note: My most profound apologies for the line that was removed. I in no way intended that remark as in any way sexist or derogatory to the abilities, aptitudes or interests of women. It was intended solely as a self-deprecating effort to poke fun at men for misusing tools. It was a poor effort at humor for which I take full responsibility.