clock menu more-arrow no yes

Filed under:

Cognitive Bias and Brandon Carlo

New, comments

How it’s becoming harder to evaluate the young defenseman

NHL: Boston Bruins at Carolina Hurricanes James Guillory-USA TODAY Sports

Data as of 3/27/18

Brandon Carlo has been a topic of confusion for many Bruins fans. Seen as a top 4 defenseman with a bright future by many Bruins fans last year, his poor play as of late has some Bruins fans doubting this.

I wrote a more in-depth evaluation of Carlo in the fall, and this is not meant to be an evaluation at all. This is a warning that your recent eye test on Carlo is filled with bias, and he hasn’t been as bad as you likely see him.

We often form our opinions of a player’s defensive play around replays of goals and scoring chances. That is only a small sample of a player’s defensive play. Any given player spends around have of their time-on-ice playing “defense.” Evaluating a player’s game around a few 10 second clips is flawed.

Now I’m not saying you only watch replays to form your opinions on players, but we all fall victim to bias in a similar fashion to that. It is one of the reasons why analytics are so important in evaluation.

An important concept to grasp is, goals act as if random. To explain that concept, imagine flipping a coin. You know that the probability of landing on heads is 50%. Whether or not the coin lands on heads is random. Consider flipping that coin 100 times. There is a very small probability that you get 50 heads and 50 tails. What you are seeing is randomness and variance.

The same thing occurs with goals in hockey. If you included all of the factors that go into whether or not a shot is a goal, you’d be given a probability of any shot becoming a goal, and what you see from there is randomness.

If you are reading this, you’ve probably heard of PDO. The combination of on-ice shooting and save percentages is an attempt to measure luck. It’s not the greatest metric, and given that shooting and save percentages are not related, I don’t look at it much. You are better off looking a shooting and save percentages separately.

Outside of Nick Holden, who has played most of his season with the Rangers, Carlo is last among Bruins defensemen who’ve played 400 minutes in goals against per 60 minutes at 5v5. Among that same pool of players, only McQuaid gives up shots at a higher rate.

Even though Carlo has clear defensive flaws, I warn that this may be over exaggerated. Carlo’s on-ice save percentage at 5v5 is last among the pool of players above at .921. When Carlo is off the ice, the Bruins on-ice save percentage is 1.19% higher, or .933.

You can read more about relative save percentages here, but what we are from Carlo is not a skill (or lack there of), but most likely randomness. Even more reason not to believe Carlo is not causing this, he has a positive impact on relative expected save percentage.

Yes Carlo is making a lot of mistakes defensively. However, because he is currently the victim of bad luck, you are seeing those mistakes more than other Bruins defensemen.