How Much Difference Do NHL Coaches Make?

The following is an analysis I first published on NHL Numbers back in November. Enjoy!

There are few things more reliable in the world of hockey than the tendency of fans to question their teams’ coaches and this NHL season is no different. Poor starts have already cost the Flyers’ Peter Laviolette and the Sabres’ Ron Rolston their jobs. Canadiens’ fans are criticizing the decision-making of Michel Therrien, and supporters of the Rangers, Canucks, and other teams have already put their head coaches under fire. On one hand, it’s all too easy to make coaches the scapegoat for problems outside their control (does anyone really believe another coach could do better than Rolston with the Sabres’ roster?). On the other hand, we’ve seen enough examples in recent seasons of systems that particular coaches are able to take with them through multiple jobs – Jacques Lemaire’s neutral zone trap being an obvious example – so it seems at least possible that one coach might be able to get better results than another from the same players. But how much does coaching matter?

Image credit: Flickr user Lauren Siegert. Use of image does not imply endorsement

In order to explore this question, I constructed a database of NHL head-coaching changes from the 2007-08 season through the 2013 season. The goal was to compare the performance of a team before and after the change in coaches, as a way of quantifying the impact of the change. In addition to the team’s season standings-point % (which I got from nhl.com) before and after the switch, I gathered their Fenwick Close %, their rates of even-strength Corsi attempts for and against per 60 minutes, and their even-strength shooting and save percentages. These latter data come from stats.hockeyanalysis.com. It should be noted that partial-season coaching performances weren’t included in my data; that is, if a coach was fired in mid-season, I didn’t include the partial season (or that of his successor) in the analysis. This was done partly to avoid bias in the analysis of point percentage, but also because partial-season advanced statistics aren’t available prior to the 2013 season. So, for example, Michel Therrien’s 2007-08 Penguins are compared against Dan Bylsma’s 2009-10 Pens, without consideration of the 2008-09 season. Moreover, coaches that failed to complete a single season are excluded entirely from the data. (Lightning fans need not worry about having to relive the horror that was Barry Melrose’s 16-game tenure as their coach.) Coaching changes that occurred after the conclusion of the 2013 season, obviously, are not included. All told, there were 41 total coaching transitions in the six seasons covered by my analysis. Five teams experienced no coaching turnover in this analysis, while fourteen had two or more coaching changes that met the criteria above.

In aggregate, the change in coaches had no clear impact on teams’ performance. The full season following a coaching change saw an average increase of 1.5 standings points from the former coach’s last season. Teams changing coaches saw their Fenwick Close % increase about 0.35%; after a coaching switch, teams averaged one more Corsi attempt for and 0.4 more Corsi attempts against per 60 minutes, and saw very slight decreases in shooting and save percentages.

Within these data, though, are some interesting case studies. In some individual instances, changing coaches had a significant impact. Specifically:

  • In terms of increasing puck possession, the best coaching changes since 2007 would be Michel Therrien to Dan Bylsma in Pittsburgh (46.3% Fenwick Close to 52.6%), Wayne Gretzky to Dave Tippett in Phoenix (45.6% to 51.2%), Marc Crawford to Terry Murray in Los Angeles (45.3% to 50.8%), Denis Savard to Joel Quenneville in Chicago (49.8% to 55.1%), and Rick Tocchet to Guy Boucher in Tampa Bay (48.0% to 53.1%).
  • How good was the switch from Therrien to Bylsma in Pittsburgh? In Bylsma’s first full season behind the Pens bench, the team was putting up 11.5 more Corsi attempts per 60 minutes, while actually allowing fewer shots than they had in 2007-08.
  • Some people are under the impression that the Kings’ transformation to a monster possession team happened in the 2011-12 season when Darryl Sutter was hired. This is only partly true. LA’s rate of Corsi attempts against dropped by 7.5 per 60 between Murray’s 2010-11 season and Sutter’s 2013, but it dropped by 6.7 attempts per 60 after Crawford’s dismissal. Both changes, however, had the effect of making the Kings a lower-event hockey team.
  • On the other hand, the worst coaching changes in terms of puck possession are Tom Renney to John Tortorella for the New York Rangers (the team’s Fenwick Close went from 55% to 49.6%), Scott Arniel to Todd Richards in Columbus (50.9% to 45.7%), and Ted Nolan to Scott Gordon for the New York Islanders (50.1% to 45.5%).
  • Gordon’s hiring by the Isles looks even worse when you consider that the team allowed over 10 more Corsi attempts against under Gordon than it had under Nolan, while actually taking fewer shots.
  • For all the Randy Carlyle fans out there: Anaheim’s Fenwick Close increased from 45.1% to 48.5% in the switch to Bruce Boudreau, and their point % increased from 60.4% to 68.8%. In Toronto, the team’s Fenwick Close dropped from 47% to 44% after Ron Wilson’s last season, while their point % increased from 51.8% to 59.4% (the team’s shooting percentage, of course, jumped from 8.4% to 10.6%).
  • Speaking of Boudreau, good hockey appears to follow the guy wherever he goes. Anaheim has been a better and more successful team since bringing him on, and the Washington Capitals have been notably worse in the switch from him to Adam Oates. Their Fenwick Close has gone from 51.0% to 48.6%, and their point % went from 65.2% to 59.4%.
  • For all the Michel Therrien fans out there: the Canadiens were a better team in 2013 than they were under Jacques Martin in 2010-11. Their Fenwick Close increased from 52% to 53.5%, a change largely accomplished by improved shot prevention.

Obviously, this analysis isn’t perfect. Significant personnel changes can happen from one season to the next, and in some cases we’re comparing a team’s performance in a season against their results two years prior. In other words, some of the changes in results are almost certainly due to roster moves. And, in general, there doesn’t appear to be a way of knowing whether a coaching change will work out for the better. In some cases, they seem to help; in others, they appear to make a team worse.

About Nick Emptage

Nicholas Emptage is the blogger behind puckprediction.com. He is an economist by trade and a Sharks fan by choice.
This entry was posted in Original Analysis and tagged , , , , , , . Bookmark the permalink.

4 Responses to How Much Difference Do NHL Coaches Make?

  1. Joe says:

    Interesting. It would be even more interesting if your analysis could account for the different reasons that coaches are hired. Some coaches are hired when the team is on the cusp of becoming a contender, while others are hired when the team is starting a rebuild.

    • Nick Emptage says:

      Very true. The sample size is the real obstacle in this analysis. I’m hoping (as we get game-level data from more seasons via Extra Skater) that some of the mid-season changes can be added to this. More generally though it might be a few more years before there are enough coaching changes to allow us to look at questions like this.

  2. Mitch says:

    Good stuff here. What I like the most is how you presented the information. Clear, concise, and without drowning me in the data. You allow me to trust the background work you put in, rather than forcing me to dig through it. As someone who is just starting to post my own work online, and trying to do so in a non-biased manner by using statistics, you set a good example on how to do so. Keep up the good work.

    • Nick Emptage says:

      Thanks very much. I do have the tendency to do those “burying people in data” posts, but I’m glad you liked this one. And keep up your own writing!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s