Evaluating players can be done in two ways (there are probably more than two, but for the sake of this intro, let's pretend there are only two): the eye test or the stat test. The eye test is a good way to do an evaluation of a players skill sets, it's an easy way to get a quick scout of a players potential and it helps you see if they understand their assignment and playing where they should be. The stats will give you more accurate interpretation of how a player and the team are doing while on the ice in terms of individual stats (goals, assists, save percentage) and possession stats (corsi, fenwick). While both of these methods are subject to some levels of confirmation bias in the way we arrive at conclusions from evaluating them, the eye test is by far more susceptible. Which is why here at BSH stats tend to find their way into every discussion (hockey based or not), because if you thought you saw it, you're likely wrong. Like the Cosby Show, your eyes lied to you (yes, that is a super deep cut 30 Rock joke).
With all that said, Kurt's recent article about Steve Mason needing rest, and the subsequent comments got me thinking. We know Steve Mason's time in Columbus wasn't exactly kind to him. Save for the first half of his rookie year he was downright awful and almost universally considered the worst starter in hockey in his 4+ years there. As far as his eye test goes, there are many people who have written on the subject. Things I notice are he's pretty big in net, he's actually pretty decently quick, and he seems to have decent flexibility. Here's the Hockey's Future Talent Analysis:
Mason is a big goalie who plays sound positionally. He is a mature, confident and extremely poised goaltender, capable of making big saves at the key times in games. Mason has a strong work ethic, and thinks the game well. He's got good lateral mobility, and plays the puck extremely well.
With all that said, I decided to do an evaluation of Mason on a season by season basis using save percentages. Because of the short season last year (and mostly because I felt like it) I also did an entire compilation of his time in Philly and his entire career. So that gives us 2009, 2010, 2011, 2012, 2013, 2014, "Flyers" and Overall. For each of these evaluation time frames I pulled his sv% data and did cumulative, rolling 5-game and rolling 10-game save percentages. In each season, for the 5-game and 10-game rolling save percentages, I pulled the Minimum and Maximum value as well as standard deviations and the total number of values above and below his career sv% of 0.906.
Now, I'm no fancy big city stat-head, but the results are below including my charts, graphs and the little further analysis I feel comfortable adding. I apologize in advance for using SnagIt copied bitmaps of excel graphs.
So, let's do a quick visual analysis of these graphs. In 2009 Mason (as we all probably are aware) started off strong and then fell off the pace and had a very poor end to the season. Optimist at the time probably said rookie burnout, pessimist would say regressing to his mean. 2010 started off with a few nice games, then tailed off and then had slight improvement as the season progressed, but all-in-all was a worse season than 2009. 2011 is probably best described as variable. He was good, he was bad, he was good, he was bad, he averaged exactly the same as the season before.
Now we're at one "good" season and two "poor" seasons. Then we hit 2012, his last full year in Columbus. This was not a good start for Mason. It takes him until roughly halfway through the season before he can put ten games together with an average above 0.900. However, if you want to look optimistically about it, his second half of this season kept getting better, comparatively . Once he got those few rolling 10 game averages above 0.900, he had a few more drop, then went back up, but then had about 12 in a row that were above. This is glass 1/4 full optimism here, but even though he ended with the worst sv% of his career that season (see chart below), it was held down by the start of his season, in almost opposite fashion as his 2009 and career sv% career were buoyed by the start of 2009. Now we move to the shortened season when he lost his starting job (to our former backup) and got traded later on(to become our starter). Very small sample size compared to other season (obviously) but, probably his best one in terms of overall sv% and rolling 5 and 10 performance, in that his lowest values were the best in his career. Basically everything I calc'ed went up as the season progressed and most due to his play here in Philly. So, that brings us to this season. Started strong, then dropped, then dropped, then kept dropping, then brought it back up, then dropped again. Depending on how his next few games go though (Hi Pittsburgh!) he could keep his rolling tens at a decent (more acceptable) but not great 0.910ish average or could continue to drop those back into the depressing "sky is falling" area that the mid-season was.
After the individual seasons, I did the overall and Flyers portions. As you can see, both show similar results. Cumulative slowing dropping and tail end of all the rolling averages also dropping. In fact his rolling averages decline pretty much his entire time with the Flyers aside from a slight (brief) rise that was due to the great play going into the Olympics.
So, in 2014 and as a Flyer, Mason's sv% is 0.913 and 0.917, respectively. Last 5 seasons, league wide save percentages have been 0.911, 0.913, 0.914, 0.912, 0.914 (current). So, if we would like to believe he has in fact improved, then we can conclude he is basically a league average goalie. When he came to the Flyers, his lifetime sv% was 0.903. It is currently at 0.906. To put that into perspective, in Columbus he faced 6553 shots and made 5919 saves. To improve his sv% by 3 points, the easiest way would have been to stop the next 192 shots straight. Which would be something like 19 straight periods of shutout hockey (~6.5 games). So, he has played "better" since coming here, but by how much is obviously debatable.
Below I have placed a table of stats from all these seasons. One thing that does stand out for me is that aside from the 2013 season, his standard deviation for rolling 5 and 10 games are mostly consistent for all seasons. Which implies to me that his level of variation is mostly the same. If you wanted to be the extreme level of optimist on that, you could say this implies he could very well be improving, albeit slowly and not by a drastic level. Basically, what I took form this was he had one really good half a season in Columbus his first year. Followed immediately by two below average with glimmers of hope seasons. Which was then followed by a frankly god-awful opening of a season with improvement towards the end that just was not nearly enough to compensate for the terribleness of the start. Then he had an above (his) average year in a shortened 2013 and continued said above average here in 2014. Of course, 2014 was buoyed by a good start, and unless he turns it back to that in the next few games he's likely to continue a downward trend. But unless it's a catastrophic final fifth of this season, he will still end up above his career average.
So, with all that said I see two real ways to be positive about this (with several ways to be negative). First, he has shown signs that he's played better here than he did in Columbus. He showed improvement at the tail end of his time in Columbus, but it was improvement on the most terrible of play that it was more likely just variation around his actual average and not improvement. But, knowing that his scouting reports all say he's got some talent, I'm choosing to hope (although not ready to believe) that he is improving slightly. Second, in his career, and in his time with the Flyers he's shown that he can string together 5 and 10 game segments with very high save percentages. So, optimistically, he is a guy who could get hot (or lucky) at the right time and play pretty out of his mind. But on that same token, he's also the guy who could repeat what he did in his only 4 games of playoff experience.
I went into this hoping that I could find something to show that there was more than just variation in Mason's play. I wanted to be an optimist about it. I'm by no means in expert in statistics, but after going through all the motions here, I just cannot conclude that he's done enough to prove he's better here than in Columbus. I guess then what this really tells me is that Mason is a mediocre goalie. He's got some visible skills and therefore capable of flashes of good to great play, but so far in his career they haven't lasted for more than a small portion of a season. If anything, he appears to be improving somewhat but probably not to the point to be anything but near average as compared to the rest of the league. In the end, this brief study really didn't do much of anything other than mostly confirm what we've already come to know.
With some luck, maybe his numbers continue to slowly rise, indicating some improvement. It will be interesting to revisit this at season end, or after next season and see if anything does change.
Editors note: Depending on what is happening with SBN here at work, I may not be able to see comments.