Rask vs. Reward: Does the performance match the pay?

Let me ask you a $56 million-dollar question, since Tuukka Rask signed his 8-year, $7 million dollar per season contract extension in 2013, how has he performed? A tough question, so let me make it easier for you, if you had to rank him, would he be “Elite”, “Above Average”, “Average”, “Below Average” or “Poor”. Easy enough?

If you’ve tuned into local sports talk radio or nightly sports programming over the last half decade, the talking heads in this town will loudly insist the Bruins longtime netminder is overpaid, not worth the contract, a choker or any combination of incendiary descriptors. Switch over to the internet and read commentary from beat writers and fans alike, insisting that Rask is above average to most, elite to some. As is often the case with polarizing topics, practically everyone offering an opinion is wrong to a degree. Some base their takes on the age old “eye test” using subjective measures like “heart”, “compete” or “battle”, whereas others selectively choose flattering statistics, consciously or unconsciously, to defend “their guy”, often as a reaction to those on the other side of the debate. Stuck in the middle are the minority of fans and media whose reasoned, nuanced voices are all too often drowned out by the sheer volume of polarization. After years of deafening opinion-based debate, let’s take a comprehensive look at how wrong many have been on Rask, and finally, mercifully allow the numbers to talk the loudest.

The current, popular statistics used to measure goaltender performance are somewhere between sub-par and nearly useless, depending on the stat in question. As such, many believe there are no stats, advanced or not, that can adequately evaluate a goaltender’s performance. While these people are correct that measuring goaltender performance is more difficult than that of skaters, they are wrong to believe better statistics do not exist. Before we get to those statistics however, let’s back up and explain why the current trifecta of goalie stats are wholly inadequate.

GAA: This has been written about in far greater detail by many others, but the point boils down to this: GAA is a team statistic and tells us virtually nothing about the goaltender’s individual performance relative to his team. How many shots did the goalie face compared to other goalies? Were those shots during special teams play? GAA doesn’t account for either of these things, among other shortcomings.

Wins: Again, see above. Wins reflect the team’s performance, not specifically the goalie’s. Logically, goalies playing behind good teams will accrue more wins, whereas those playing behind worse teams will accrue less. Further, a goalie cannot affect his team’s performance on offense, so judging a goalie by a true team stat, is by nature flawed from the start.

Save %: Save percentage is the statistic used most often by fans and media to evaluate goalies. Unfortunately, it is still inherently flawed as it lacks virtually any context. It doesn’t factor in game state (PK/PP, etc.), nor does it consider the quality, location or danger of shots the goaltender faced (Low, Med, High danger). Save percentage has value as a low-level snapshot but inferring anything more significant than that should be avoided, and it should not be the primary statistic used to evaluate goalies in 2018.

The common theme is that these frequently used statistics fail to isolate the most important aspect: the goaltender’s performance separate from the team’s performance in front of him. Easier said than done though, right? Of course, but there have been numerous advancements in the world of hockey statistics that are able to accomplish this, to varying degrees. If you’ve made it this far, I assume you are following along, hopefully more or less without issue. Next, we get to the part where many fans politely remind us that “the game isn’t played on a spreadsheet, nerd”. If you happen to be one of those fans, you will likely hate the rest of this article, which is a shame because there is so much to be gained from using ever advancing statistics to evaluate players.

Methodology: I’ve used the timeframe since Rask signed his current contract (2013-14 through present) to frame the discussion. At the core of the debate, Rask is paid like a starter, so he should be benchmarked by other starting goalies. For that reason, I’ve also isolated the NHL goalie pool down to “starters” by removing any goalie who recorded less than 2,000 5v5 regular season minutes per season (minimum of 10,000 total 5v5 minutes for 2013-18 stats). This helps account for injuries and other factors which gives a better peer group of actual “starting quality” goalies, instead of including career backups and injury call-ups. I’ve used only regular season, 5v5 statistics throughout, as is standard when using hockey statistics, as 5v5 is the most common game state and allows us to compare like to like things. For those new to hockey statistics, the Athletic’s Charlie O’Connor covered these topics and more in his excellent advanced stats primer which is an excellent, intuitive description of how to utilize hockey statistics for those unfamiliar.

I’ve specifically chosen 7 statistics, each of which has been tested and/or peer reviewed and shown to be the among the most accurate, repeatable and/or useful of those currently available publicly. They are all explained in layman’s terms here and in further detail in the links I’ve included throughout. If you aren’t familiar with these statistics, I highly recommend reading up on them. Most are more intuitive (if explained correctly) than many may assume.

You will see a “Z Score” for each 2013-18 statistical result, which is a way to provide context around result distribution, in addition to Rask’s rankings throughout. A Z score is simply the number of standards deviations the result (Rask) was from the mean (weighted average), which is important for results like save percentages that tend to cluster together. For example, a hypothetical goalie may rank 20^th out of 25^th in Sv% but may only be a negligible amount worse than most goalies above him. Z scores provide a contextual degree of performance by telling us how much better or worse a player was, relatively, than the weighted average. Keep in mind a negative number means a player was worse than average and a Z score of 1.0 would be a full standard deviation from the mean.

Finally, I have weighted each statistic average by shots against, except for GSAA which already incorporates shots faced, which I weighted by 5v5 TOI. All underlying statistics come from Emmanuel Perry’s Corsica.Hockey.com.

Statistic: Low Danger Save Percentage or LDSv%

The least valuable but most repeatable of the statistics referenced in this article, LDSv% is the percentage of saves a goalie makes on shots from “low danger” areas. An in-depth breakdown of how danger areas are classified can be found here. The simplified terminology is “Save percentage on shots with a Fenwick shooting percentage of 3% or under”, where Fenwick shooting percentage is the shooting percentage of all unblocked shot attempts. Every NHL goalie should be able to stop the majority of low danger shots (and largely do), hence the relatively close distribution of results. As such, it tells us less about the skill level of the goaltender in question, but rather how often a goalie lets in “easy” goals.

Results: As the graph shows, Rask has been above average to elite at stopping low danger shots, relative to his peers, as evidenced by his Z score of 1.23.

Statistic: Medium Danger Save Percentage or MDSv%

With Medium Danger Save percentage, the Fenwick shooting percentage is above 3% (Low danger) and equal to or less than 9% (High danger). Like low danger Sv%, NHL goaltenders tend to stop most medium dangers shots. As Emmanuel Perry of Corsica put it in his explanation, “it appears the skill-driven component of Sv% is almost entirely contained in a goalie’s ability to stop shots of the High-Danger variety”. As with LDSv%, MDSv% has value but is still not crucial to evaluating a player other than to provide helpful context.

Results: Whereas Rask was bordering on “elite” at stopping easy shots, he has been below average in relation to his “starting” peers on mid danger shots, as shown by his Z score of -0.62.

Statistic: Save Percentage or Sv%

As previously mentioned, Sv% is a flawed but commonly used stat, and it is needed to provide context for our next two statistics, so follow along.

Results: Rask has been average to above average in terms of save %, as his 8^th place ranking and 0.68 Z score over the last 5 seasons shows, however he has been steadily declining since his strong 2013-14 season. Keep in mind that raw save percentage shouldn’t carry much weight on its own.

Statistic: Expected Save Percentage or xSv%

Here is where we start to get into statistics that many may not be familiar with but bear with me. Expected save percentage is a statistical model that incorporates various inputs, including the probability of each shot being a goal, as a way to consider the team’s effect on a goaltender. The model uses this information to provide a save percentage the goaltender SHOULD have with the team in front of him. It is more complicated than that, as detailed here, but for our purposes the higher the xSv% number , the “easier” a goalie has it. This is an example of a “predictive” stat, rather than a “descriptive” stat like raw Sv%.

Results: This shows us that Tuukka Rask has had among the “easiest” workloads in the NHL over the last 5 seasons, relative to his “starting” peers, as shown by his Z score of 1.57. Essentially, the Bruins defense largely kept scoring chances outside of high danger areas, limiting the high danger shots he faces. Contrary to widespread belief, the Bruins defense has been among the best in the league over the last half decade. This is evidenced by top five NHL rankings in CA/60 (Corsi Against per 60 minutes), xGA/60 (Expected Goals Against), and even Shots against/60, as well as more basic statistics like goals against, penalty kill %, etc. Now that we know Tuukka Rask SHOULD have performed at a near Elite or even Elite level, the logical next step is to compare the “predictive” expected save % to the “descriptive” actual save %.

Statistic: Delta Save Percentage or dSv%

Delta or “adjusted” save percentage is the difference between the expected save % and the real world save %. In essence, dSv% shows us how much better or worse a goalie did than expected. It is what save percentage should be, and frankly should (and may) start to phase out Sv% from most broadcasts, websites, etc. in the next 5-10 years. In terms of importance, dSv% and the following two statistics are the most significant and should be weighted appropriately.

Results: Rask has been negative in 3 of 5 years, as well as overall, with a Z Score of -0.77. This means that not only has he done worse than the hypothetical average “starter” would have (represented by the blue bars), he also has done worse than the true average (“0” on the horizontal axis) which represents all NHL goalies. This, despite being paid like an elite goaltender. Over the last 5 seasons, he ranked 20^th out of 25 “starting” goaltenders. In plain English, by this metric, he has been quantifiably below average for a starting goalie.

Statistic: Goals Saved Above Average or GSAA

GSAA compares a goalie to a hypothetical “league average” goalie and then counts how many goals that goalie either saved or cost his team in comparison to the league average goalie. GSAA is starting to be used across traditional media and broadcasts and as such it is a statistic hockey fans should familiarize themselves with. While not the same, GSAA is like the baseball stat “WAR” or “Wins Above Replacement”, in that it presents the information in tangible terms (actual goals saved/cost) instead of a percentage. For context, a goalie with a GSAA of 0 is exactly average. As with any counting stat, the amount of TOI a player receives directly affects their results, which can be standardized by using GSAA/60 or GSAA/30 to account for differences in TOI. For those interested, GSAA is explained in further detail here and here.

Results: As with dSv%, Rask has been worse than the average “starter” in 3 of 5 years, as well as overall. Expressed in goals, over the past 5 seasons, Tuukka Rask has cost the Bruins 7.29 goals more than a “league average” goalie (not just starters) would have. Compared to the “starter” weighted average, Rask has cost his team 19.85 goals since 2013. Since GSAA is a counting stat, the 2013-18 numbers are much higher/lower than the single season results. Another way to interpret this stat is “how dependent is a team on their goalie for success”, where Columbus, Colorado, etc. all relied heavily on their goalie to play well above average or “bail them out”. This is not to say that Rask hasn’t had individual games where he bailed out the Bruins, rather, overall it shows that the Bruins are less dependent on their starter for their success than average. His -0.79 Z score indicates that he is again below average and has slightly hurt the team.

Statistic: High Danger Save Percentage or HDSv%

As referenced in the LDSv% section, HDSv% is currently the best statistical approximation of a goalie’s actual skill level or “talent”, and as such should be considered more significant than LDSv% and MDSv%. As previously mentioned, it is based on Fenwick shooting percentage, in this case any shot over 9%. This is essentially any area on the ice that results in a higher than league average shooting % result.

Results: How many times have you heard that Rask is a “system goalie”, or that he is the “Anti-Thomas” in terms of playing style over the years? At times it’s been hard to escape if you live in New England, and unfortunately this chart is not going to help change that perception. Before I started researching this article I asked myself the same question I started the article with. My answer? I felt that Rask had been largely average since signing a deal that made him the 3^rd highest paid goalie in the NHL. Specifically, I felt that he struggled with less routine stops as a goalie who relies significantly more on positioning and efficiency than on pure athletic skill or size. The numbers seem to agree with my unscientific eye test as Rask has been below average, bordering on poor, at stopping difficult shots, as shown by his Z Score of -1.10.

---

Takeaway: The best available stats tell us Rask was good at stopping easy shots, not so much with harder ones and was rarely above average (either year by year or overall) in any meaningful advanced statistic. In fact, he has been regularly below average and in some cases among the bottom third of starting goaltenders. Worse yet for Bruins fans, he is seemingly getting worse each year, despite minimal changes to the team defense in front of him, as one can reasonably expect due to goaltender aging curves.

Before you grab your pitchforks, remember that no statistic or combination of statistics is 100% accurate, however the more advanced ones do a very good job of getting us close. The stats plainly say Tuukka has been a below average starting goalie but if for example, you wanted to argue Rask is average based on the value of all situational play (his stats are slightly better vs. 5v5), or that the available stats overestimate the strength of the defensive play in front of him, I won’t stop you. I might argue that if you can’t specifically pinpoint what the statistic fails to account for, you should be slow to dismiss the results, but at the end of the day there will always be some amount of leeway in a sport as complex as hockey. Crucially, none of this means Rask cannot rebound or “return to form” as goalies are notoriously prone to extreme variance in their results from year to year. However, his age and recent history tell us that even though it’s certainly possible, it isn’t particularly likely.

While many could have predicted that the deal wouldn’t work out in the long run, that doesn’t mean the Bruins should not have re-signed Rask altogether. Their biggest error was in letting Rask bet on himself. Rask’s bet succeeded spectacularly, resulting in a massive contract. A contract it should be noted, that entering its 6^th season, still sees Rask comfortably among the NHL’s five highest paid goaltenders. The fact of the matter is even though signing goaltenders to expensive, long term deals rarely works, the Bruins didn’t have much of a choice. They have notably struggled to groom a young successor to Rask, and quality starting goaltenders aren’t often available via free agency or trade. Despite his lackluster performance since re-signing, Rask was in-arguably one of the best goalies in the league at that point in time. The NHL market is inefficient, rewarding aging, declining players over young improving players. GMs overpaying for goalies will continue to be the standard, at least until GMs start to catch on, as Yzerman did with Andrei Vasilevskiy/Ben Bishop, or Jim Rutherford did with Matt Murray/Marc Andre Fleury.

So, ready to answer to our $56 million-dollar question? Here goes: is Tuukka Rask elite? Absolutely not. To borrow a common refrain, does that mean he “sucks”? Absolutely not. At the time of the deal had he earned it? Absolutely. Is he, and has he been overpaid relative to his actual performance ever since? Absolutely. Contrary to common belief, all these statements can exist together without being in conflict.

The Bruins, for better or worse, are stuck with 3 more seasons of Rask’s expensive, partial No-Trade Clause laden contract. Even if Rask agreed to waive his M-NTC, a rare occurrence in the NHL, he would hardly have significant trade value, nor do the Bruins have the organizational depth needed to move on from him. The Bruins $56 million-dollar man has statistically been somewhere between average and below average. Where he falls in that range is largely up to your own subjective viewpoint. At the end of the day, whether Rask earned his contract prior to re-signing isn’t truly up for debate. However, after 5 seasons of relative mediocrity under the growing shadow of its lofty expectations, the answer to whether he will ever live up to it is becoming increasingly clearer for fans and management alike. Entering the 6^th year of his 8-year contract, the Bruins must be hoping Rask’s friend and former partner Tim Thomas can graciously share the secret to a late career resurgence. Without it the Bruins will only go as far as Rask takes them, and Rask doesn’t seem to be going anywhere soon.