Brodeur

 

As the man with the most wins and the most shutouts in history, Martin Brodeur is often talked about as the best goaltender ever.

 

But Darcy Norman, author of Hockeynomics, emphatically disagrees.

 

Norman played at the junior level before heading to the University of Alberta to earn his degree in economics.  A self-professed lover of the game, Norman believes numerical rigor can illuminate the many still-dim corners of the hockey world.   In his book, Hockeynomics: What the Stats Really Reveal, he brings together the views of some of the most successful personalities in sports statistical analysis to answer questions like:

 

  • Who’s good at the draft? (and Hockeytown does not fare as well as you might think)
  • Who will have the better career, Crosby or Ovechkin?
  • Which teams spend wisely?
  • Was Gretzky’s 92 goals in 1981-82 the finest goal-scoring performance ever?
  • Is Martin Brodeur overrated?

All of these questions Norman addresses with heavyweight mathematical theory, but his ease with the concepts and his casual writing style keeps the material accessible (and kept my eyes glued to the pages deep into the night).

 

It is the investigation of this last question concerning Brodeur that I want to share with you, because while many of the arguments in Hockeynomics have fantasy uses, the examination of Brodeur results in two new statistical measures that I think are relevant to goalies and their fantasy owners.  And I’d love to see more of them.

 

Is Martin Brodeur the Best Goalie Ever?


While most of us agree that save percentage is a better statistical measure of a goalie’s ability than goal-against-average (and if you don’t, you should), Norman shows us how to take the concept of save percentage even further.

 

In answering the question of whether Brodeur is history’s best goaltender, Norman builds his case around something that many people already believe to be true: while the Devils have been stingy with the quantity of shots surrendered over much of Brodeur's stellar career, they've also been stingy with the quality of shots allowed.

 

In order to prove this, he introduces the reader to the concepts of Shot Quality Against (SQA) and Shot Quality Neutral Save Percentage (SQNSV).

 

Slightly scary terms, I know, but hang in there.

 

SQA begins with calculating the likelihood of scoring on different types of shots (Norman doesn’t overwhelm us with the details here, but points out that most of us can appreciate that a slapshot from 10 feet out has a higher likelihood of producing a goal than a weak backhand from 30 feet out).  Once that is catalogued (no small amount of effort there), a league average can be compiled and then combining that with the number of shots a team faced allows you to predict a number of goals scored against for that team.  If that team gives up fewer goals than expected, it is assumed that they gave up fewer quality scoring chances, and vice-versa.

 

SQA can be calculated for a team, or an individual goalie (by only using games in which a particular goalie played).  Any guesses how the Devils made out?   It turns out that in recent history their SQA is very low, meaning that they give up far fewer quality shots than the rest of the league.

 

SQNSV is calculated by applying SQA to a team or individual goalie’s save percentage, therefore producing a version of save percentage that factors in the difficulty of shot faced.

 

By using SQA and SQNSV it allows for analysts to peel back one more layer of the team element when determining the individual abilities of a goalie.



The results for Brodeur?  Not great.

 

The Verdict on Brodeur’s Place in History



Seasons 2002-03 through 2008-09 were unimpressive when viewed through these new statistical measures, "Brodeur failed to lead the league - ever- in adjusted save percentage (SQNSV)...in fact, Brodeur only cracked the top 10 in the statistic once, in 2007-08.  Does that sound like the resume of someone considered as the best goalie of all time?"



Norman does acknowledge that Brodeur is "an extremely good, if not great, NHL goaltender.  However, whereas he does exceptionally well in team-based counting statistics, like wins and shutouts, he is pretty average when it comes to individual performance measures.  Where Brodeur really shines is in durability; between 1993-94 and 2008-09, he played 995 games.  That's 226 more games than the goalie with the next highest total over the same time span.  That kind of stable, predictable performance is hugely valuable."

 

So there you have it, “hugely valuable”, but not “the best”.

 

Beyond any debate focusing on Brodeur, I’d love to see the SQA and SQNSV measures used more often, because they would certainly help us poolies get a better handle on goalies’ specific strengths and weaknesses which would then allow us to make better predictions when circumstances surrounding those goalies inevitably evolve.



If all of this made you angry/upset/strangely irritated, well, then head on over to the ramblings and I won’t take offence, but don’t forget your ‘I heart Brodeur t-shirt’.

 

If you do want to know more about what Norman is getting at, then check Hockeynomics (I’ve see it at Chapters and Indigo stores as well as in various airport bookstores around Canada).  Or explore this site or this one for some hardcore number play.

 


Write comment
Comments (31)add comment

Repent Tokyo said:

repenttokyo
the most flawed data in the nhl is hits and blocked shots, due to the use of team employees instead of NHL personnel to collect it.
November 27, 2010
Votes: +0

Larry said:

Rollie1967
more info is always a win Interesting info, again there are so many intangibles that go into 'greatness',but any info that sparks debate im all for.
It was just in the mid 70's when a similar debate was common regarding Ken Dryden on the Habs vs the other top goalies of his prime:Resch,Parent,Vachon,Palmateer,Esposito. Sure Dryden didnt face as many shots as some of his peers (Vachon never had a good team in front),but most goalies will say facing fewer shots is tougher. How do you quantify dealing with the pressure of playing in Montreal,NY,Toronto vs knowing you have Lafleur,Shutt,Lemaire,Cournoyer to bail you out,give you a cusion?

Im still waiting for the NHL to adopt Roger Neilsons concept for powerplay time required to score vs what we have now that considers a 2 second opportuinity the same as 5 minutes. With all the stats/data we have in major sports today-that might be the most flawed.
November 27, 2010
Votes: +0

Darcy Norman said:

darcynorman
re: Ed Murdoch Hey Ed,

Separating team defense from goaltending is extremely difficult to do. In fact, it’s difficult even when watching a game, live (though I’m sure some fans would disagree). The problem is due to the dynamic flow of hockey, there aren’t many measurements. But it’s all about logic; think about how YOU would do it. When you have some idea, try to see if the data exists to help you out. That’s how the clever analysts did it with shot quality. And when you think about it, it makes sense: Once you know the difficulty of any shot, you can calculate whether a team defense is above or below average. And as Alan Ryder (who pioneered the work) says in the paper, “if defense is better than we thought, then goaltending must be worse (and vice versa)”. That might be hard to swallow, but what he means is, if two goalies have a .929 save percentage, but one plays behind a terrible defense (give up higher quality shots), then that goaltender must be better than his stats indicate.

As far as putting someone else in Brodeur’s skates, we DO have a proxy for that: his back up. Granted, that’s a feat with Brodeur because he’s so damn durable (which I argue is a huge chunk of his value). But, you can compare how his backups have performed in and out of New Jersey to get an idea how the Devils can (well, could. Not so much anymore) inflate a goalie’s stats. Check them out for yourself and see what you think. Crude, but effective. Marginal information gains are what it's all about.

-Darcy Norman
November 25, 2010
Votes: +0

moosesinhooses said:

moosesinhooses
Good Stat A good peripheral stat to simulate how goalies would do on different teams and it seems a good way to quantify team defence.

To argue statistically about the "best" goalie ever, you need to define "best" by a combination of stats, not just achievements (of which Martin has many). Until then you are attacking/defending apples with bananas......and any Monty Python fan can tell you how to defend yourself against a man armed with a banana (+-5%, 99 times/100)
November 25, 2010
Votes: +0

Ed Murdoch said:

emurdoch
A Goalie's Job All very good points Darcy and I'm impressed that you're willing to provide more information on the background of your analysis. There are many factors that can affect a goalie, and it certainly gets messy when you consider that different goalies would react differently given the same situations.

The defence might limit the opposition to 3 shots per period.. which could be a blessing or a nightmare depending on who you put between the pipes...

So, where I wholeheartedly agree that "a team's defense can strongly influence how hard a goalie's job is"... How can we determine which way the influence will tilt?

I sometimes wonder if it would be possible to put another goalie into Brodeur's skates and see how the numbers would translate...
November 25, 2010
Votes: +0

shook81 said:

shook81
Awesome Article... Confirms what I've suspected for years! Thanks for the great info, and I'll be looking up this book immediately...
November 24, 2010
Votes: +0

Darcy Norman said:

darcynorman
Best ever? And since it's been asked a couple of times:

The analysis isn't really capable of determining "best ever", since it depends on play by play data, which is relatively new. That leaves us stranded with recent history. Off the top of my head, Luongo and Tomas Vokoun were consistently solid performers. Oddly, they played on teams on the opposite ends of the spectrum for much of the analysis; Luongo getting peppered on a bad Florida team, Vokoun playing behind a solid defense in Nashville.
November 24, 2010
Votes: +0

Darcy Norman said:

darcynorman
re: Repent Tokyo Hey Tokyo,

I didn't "invent" any criteria, standard criteria were merely expanded upon. Just like how at one point we didn't record ice time, but now we do. This allows us to look at two players with similar statistics and say, "Hey these players scored the same amount of goals, but this guy did it in one half the ice time". That's relevant, no? As we record more and more data, we have to put it to use.

How do you choose to determine the quality of a hockey player, in this case a goalie? Is it wins? Is it save percentage? Is it goals against average? Each of those measurements has noise in it, with the most prominent being that a team's defense can strongly influence how hard a goalie's job is. The purpose of deeper analysis is to try to tease out some of the outside influence, and get that much closer to the truth. When you do that, sometimes what you find flies in the face of "common knowledge". None of the metrics used in the book are perfect, but many were developed by some smart people with a stake in the game (you mention Corsi numbers).

So, implicitly we ARE arguing about the merits of statistical analysis, as it appears you assume that the measurements we have are good enough without ever having read the methodology and logic behind things like SQNSV (which have been published free on the internet).

Again, there is/was no agenda. Brodeur being "overrated" was simply a product of people looking a little deeper into how we rate goalies. There are legions of fans who find that kind of digging unecessary and harmful to their enjoyment of the game. That's fine. On the other hand, there are others whose careers depend on finding an edge in the data. Hockey is a business after all, and getting more performance per dollar is crucial in a salary cap world.

As far as marketing for the book, well, too little too late, as the book is already over a year old. But I'll take what I can get smilies/smiley.gif

Cheers,
-Darcy Norman
November 24, 2010
Votes: +0

doulos said:

doulos
Team Defense and Goaltending Just for the sake of argument, how can we account for the fact that maybe Brodeur was good enough to play well under the New Jersey style of play. This is statistical analysis that doesn't allow for any "control" factor here because we can't compare Brodeur in his prime on hew Devils team compared to Broduer in those same years on other teams.

So, I think the reason why these sorts of discussions are interesting and great for speculation - and not of any real practical value - is that they cannot be tested in much of a real scientific way.

Keep it up though, it's good to look at and say "Hmm, interesting."
November 24, 2010
Votes: +0

David said:

OceanMon
WHO IS THE GREATEST?!!? Who is the greatest goalie of all time according to this stat then???
November 24, 2010
Votes: +0

Repent Tokyo said:

repenttokyo
also in terms of it being eons ahead of treating all shots equally - well, not if you are analyzing patrick lalime...
November 24, 2010
Votes: +0

Repent Tokyo said:

repenttokyo
you say that there is no point in debating the merits of "statistical analysis" but that is not what commenters have been debating. rather, they have been questioning the importance of the stats chosen by the writer to play such a prominent role in the analysis of brodeur posted above.
November 24, 2010
Votes: +0

Darcy Norman said:

darcynorman
Comments Hey Brent,

Thanks for the post, glad you enjoyed the book.

To be clear, SQNSV was a metric originally developed by Alan Ryder, with great work done in the area of shot quality done by both he and Ken Krzywicki. Google them if you have any interest in this area of hockey.

Someone in the thread asked how, specifically, shot quality was calculated. This was done by parsing the play by play data and segmenting the shots according to distance, and in some cases, type along with their propensity to go in the net. So, a rebound on the doorstep is a "higher quality" shot than a shot taken as the player crosses the blue line. It's a bit crude, but is eons ahead of treating all shots equally.

People who dismiss the validity of a metric like SQA or SQNSV forget one thing: if we don't account for it, then we're UNDER rating a team's (in this case, New Jersey) defense. In other words, what slightly diminishes Brodeur's statistics is a testament to how good the Devils played D.

Finally, there isn't a whole lot of point arguing the merits of statistical analysis. Some people get it, some don't. But sometimes the intention of statheads is misinterpreted. The purpose of deep analysis isn't to prove that intangibles don't exist, but rather to understand the tangible first. And as far as hockey goes, there's much to be learned from what we do measure before we're close to needing to understand the things we don't.

Cheers,
-Darcy Norman
November 24, 2010
Votes: +0

doulos said:

doulos
Interesting I found it an intriguing read but like many of the other "metrics" out there I think they stay in the realm of interesting conversation starters and don't go beyond that.

Is Brodeur the best ever? Impossible to say. But having watched him play over the course of his entire career, what he has accomplished, and the way he has accomplished it, are nothing short of stunning.

November 24, 2010
Votes: +0

sheldon said:

November 24, 2010
Votes: +0

sheldon said:

golfguy1672
... All I can say is I bet lidstrom, Pronger, mcinnis, bourque, etc for all star goalies would love to argue that their 30 foot + shots weren't irrelevant lol.


Good read, but kind of a puff piece. Lol.
November 24, 2010
Votes: +0

David said:

Leumas
... "If that team gives up fewer goals than expected, it is ASSUMED that they gave up fewer quality scoring chances, and vice-versa."

Umm....can this really be assumed? So many outside variables.
November 24, 2010
Votes: +0

Repent Tokyo said:

repenttokyo
O-Kay So, basically, this author invents his own unique statistical criteria for hockey, and then judges athletes based on these criteria. Sounds a lot like how Corsi numbers were developed, and we all know how well that concept was embraced by the sport.

It's a great way to be controversial and sell books, of course. I have no problem with different points of view on the sport, they're fun to read, but this is great marketing for his book.
November 24, 2010
Votes: +0

Victor Deng said:

Victagious1
... Definitely agree with this. Same reason why Columbus and Ken Hitchcock's defensive system inflated Pascal Leclaire and Steve Mason's stats respectively during his tenure. The quality of saves against is lower on a team that plays tighter defensively as most shots come from the halfwall or outside the prime scoring areas.
November 24, 2010
Votes: +1

Kyle said:

NJDev803
... Ok...

So who is the best? It has to be someone, right? Especially considering that all impartiality has been removed and this is strictly a study of numbers.
November 24, 2010
Votes: +1

r22yu said:

BlackGold
He's still a Hall of Famer Great article, I like the statistical analysis approach on hockey.

But this still won't change the fact that Marty is going to go down in history as the greatest goalie to grace the NHL, or take away all the goaltending records that he holds. Only Lundqvist could chip away at that smilies/tongue.gif
November 24, 2010
Votes: +0

barneyg said:

cabro57
Another team effect I'm curious as to whether the book has anything on another team-based effect on Brodeur's SV%. NJ has been one of the least penalized teams for most of Brodeur's career. For every goalie, SV% is much higher at even strength than on the PK, so goalies that spend less time on the PK (such as Brodeur) will have a higher SV%.
November 24, 2010
Votes: +0

Ed Murdoch said:

emurdoch
Stats I think I agree with the general feeling that the author seems to be discounting the stats which show greatness in favour of the stats which show otherwise. Numbers are tricky and it's easy to allow personal preferences to fill in the gaps.
Does facing fewer quality shots make a goalies job easier or harder? You can make an assumption, but it's not measurable with the extra variables given.
Maybe Brodeur was in the right place at the right time with the right balance of skills and mental fortitude for the situations he was given. If so, does that positively or negatively impact the results?
November 24, 2010
Votes: +0

sentium said:

Dakkster
... This reminds me of any blogpost written by PuckStopsHere over at KK. Completely ridiculous, that is. Big meh, not for me at all.
November 24, 2010
Votes: +0

notoriousjim said:

notoriousjim
... This feels like a lot of the SABRmetrics that really took baseball by storm over the past 10-20 years. THis seems like a reasonable measure to isolate the goalie away from what the rest of his team does. While in baseball this is simpler since it is normally only a pitcher vs hitter dynamic or a fielder vs ball dynamic, most other sports have too many other moving parts to really evaluate this.

This makes me think of park neutral ERA. While it does normalize stats and prove that Colorado does not have the worst pitchers ever during the steroid era, it does miss lots of other aspects of the game. The way a team comes to the plate against a team they think they can go yard on is different than the way they approach Nolan Ryan.

Hockey does the same thing, Brodeur makes everyone around him better. THe confidence his team gains, and the fact that most teams will not take bad shots since they know they have a wall in front of the net. That means that brodeur does less than others and does not need as many saves. THat irregular shot on goal, can hurt his ability to get into a rythm. It is the same as a pitcher on a roll leaving a game after a rain delay. You have to refocus and keep limber.

I am not a fan of some metrics, and this one seems a little outlandish.
November 24, 2010
Votes: +0

Todd Whittington said:

leaftodd
Brent My commentary wasn't aimed at your article Brent. I enjoyed it and will buy this guys book and see what he has to say. I'm quite curious about his overall views and how they sit with my beliefs.

Thanks for bringing his book up and pointing out what may be the most galvanizing theory he espouses for us to belittle.

I quite enjoyed the article.
November 24, 2010
Votes: +0

SirKnight said:

SirKnight
Flawed Analysis. There's a coaching philosophy in the NHL that has been around forever (or at least since I began watching games in the early 80s) that defeats the usefullness of the new stat SQA. A goalie makes some big saves in the first period and during the first intermission the coach comes in and says "throw everything at the net." SQA is suddenly reduced and expectedly so, because of the perceived quality of the goaltender. It can not therefore be used logically to determine individual greatness. How many times did teams go into New Jersey and before the game even started the coach says, "We are playing the best Goalie in the league, THROW EVEYTHING at the net!"? Further to that, the goalie who makes all the saves is infinitely more valuable than the goalie who makes all the quality saves but lets the occasional weak backhand shot from 30 feet out, in. If the only way Brodeur's greatness can be challenged is through the creation of new statistics, I would say, that fact alone, speaks to how great he truly is.
November 24, 2010
Votes: +0

Todd Whittington said:

leaftodd
... I'm cynical as to his method of determining what is a quality scoring chance. Not that i disagree that Marty has been the beneficiary of some great defensive teams, but to say he can effectively go back and measure what the quality level is on every shot over the past say decade is not only a huge undertaking but its also a completely biased one. Unless he looked physically at every single shot taken on every goaltender, how could he build an average for the league? Or how could he even build one goalies ratio?

Sure a 30 foot backhand is a weaker shot that a 10 foot slapshot, but what if there's a screen on the backhand? What if the goalie was physically interfered with (crease crashed)? There are so many factors involved in qualifying what is a good scoring opportunity that weren't tracked over time at every establishment.

More so how about the quality scoring chances that were missed nets and posts because a goalie was in a position that the shooter was forced to shoot at too tight a target and missed completely. That is the bread and butted of a strong positional goalie.

I'm going to pick this book up and give it a read through over Christmas because i love new theory's but to me you cant boil down who was or wasn't the best or even top 10 to a bunch of statistics well after the fact.

Just my 2c
November 24, 2010
Votes: +1

thunderbay35 said:

rtstr
...
this is really interesting, but i think at the same time it's kind of dumb. you can't determine who the "best goalie of all-time" is according to stats. maybe the best "fantasy" goalie of all time, but these guys aren't playing fantasy.

it comes down to so much more than stats, no matter how you crunch them. it's the level of confidence a team has with you back there, your ability to make that big save when it counts, the ability to consistently compete on a higher level than everyone else... that's what makes Marty who he is. it's all intangibles.
i'm not saying that he's the best goalie of all-time, but I could just as easily write an article denouncing this whole method of judging a player, so i think its kind of cheap to present it in a way that suggests it actually implies something real and useful. watch some of "The Goalie Gurus" videos on youtube. that guy is an absolute clown, and this kind of reminds me of that.
November 24, 2010
Votes: +0

Shao said:

Shao
... Thanks for the article. Beyond the numbers is important to statistical arguments. Numbers do lie.
November 24, 2010
Votes: +0

Ryan Van Horne said:

Scribe
Good column Thanks for this, Brent. Nice read. Going to have to pick up that book, too, or put it on my list for Christmas.
November 24, 2010
Votes: +0
You must be logged in to post a comment. Please register if you do not have an account yet.

busy