The Mariners and Catcher Defense
Think back to the heady days of 2010. The M’s were wallowing through a soul-crushing season after the false-dawn that was 2009. The M’s had two youngsters manning the catcher position that year, Rob Johnson and Adam Moore. They’d moved former #1 prospect Jeff Clement to Pittsburgh the year before, getting Jack Wilson and Ian Snell in a deal that somehow everyone lost. The M’s youngsters weren’t playing to bolster the offense – they combined to hit…you know, *don’t* look it up. It’s actually worse than you remember. For a brief moment, Moore had some offensive promise, but given the trade and Clement’s poor defensive reputation, the M’s implied that Johnson/Moore had MLB-quality defense (given that neither would’ve been projected to out-hit Clement).
Even in 2009, the M’s seemed to see something in Johnson’s receiving that got him playing time in front of the established Kenji Johjima and the “can’t miss” prospect, Jeff Clement. Some pitchers complained about Kenji Johjima’s receiving, but we had no way to quantify that. What we *could* quantify told us that Rob Johnson wasn’t exactly Ivan Rodriguez. His SB/CS% was a step behind Johjima’s. His PBs/WP allowed were hide-your-eyes terrible. The less said about his batting, the better. It’s like the M’s somehow believed there was an absolutely massive, yet hitherto un-measurable trait catchers had that could both influence runs scored, and the magnitude of which *dwarfed* things like passed-balls and caught-stealing. It seemed so bizarre.
Enter Mike Fast, Max Marchi and Dan Turkenkopf (with Marchi’s hiring this year, all of them are now in the employ of big league teams). Turkenkopf’s initial 2008 study of a skill called catcher “framing” made a lot of waves, but even in sabermetric circles, it seemed far-fetched. The idea that Kenji Johjima gave up 0.6 runs *per game* with his receiving seemed completely impossible. Our own Matthew Carruth investigated this through the Johnson-vs-Johjima lens in 2010. Then Mike Fast’s definitive 2011 study came along, which was reenforced by Max Marchi’s whatever-is-beyond-definitive work later in the year. Each of these studies of framing found that certain catchers were clearly better, year after year, than others, and that the effect was huge – many multiples of the range in, say, controlling the running game or blocking pitches. So the M’s were right then? Well….
It’s early, but 2014’s definitive study of catcher framing came out this week at BP, as Dan Brooks and Harry Pavlidis (of Brooks Baseball/Pitch Info) used a new method to control for pitcher, count, and (crucially) pitch type. That they found the same size effect is no longer noteworthy. But they also included tables of the best/worst framers in total and on a rate basis for the pitch fx era (2008-2013). Of the 10 worst pitch framers of the past half-decade, HALF played for the Mariners. Five. Rob Johnson comes in at 3rd worst at -26 runs per season, saved by John Hester and the most consistently-cited “bad framer,” Ryan Doumit. Jesus Montero comes in behind Johnson, and he’s followed by Johjima. Adam Moore’s at -22.5, and John Jaso comes in 10th at -20.8.
There are any number of objections to this – none of these guys actually got a full season behind the plate. The M’s saw Jaso and Montero were poor defenders, so they moved them across the diamond or across the division. And hey, what about Miguel Olivo? He’s not on the list. It’s true – he’s not. But he checked in at -12.2 runs in his half-season in 2013, and I think you’ll all believe me if I say that the new metric doesn’t rate his pitch blocking very highly. You can lodge a meta-critique and say that the entire business of assigning such large run values to catchers is suspect, especially when it’s hard to track in actual runs allowed. This is either damning, or way overblown, so forget the numbers – you don’t have to believe that basically every catcher in the past 5 years took away more runs than Brad Miller added above replacement overall. Forget run values for asecond though: how could *everyone* the M’s trotted out between Yorvit Torrealba and Mike Zunino rank so poorly?
My point in this isn’t to blast M’s management for being behind the times, or for poor player development, or any of that. The problem pretty clearly spanned both the Zduriencik and Bavasi regimes. I’m legitimately curious as to how results like this could happen, particularly in an organization that seems to value defense overall and defense from the catcher’s spot more than most. It’s not mystifying that they were frustrated with Jaso or Montero. It’s mystifying that they grab an Olivo or, yes, a John Buck to fix the problem (John Buck ranked dead last in framing runs in MLB in 2010, and finished 3rd from the bottom last year).
So clearly, the M’s don’t put too much stock in these new-fangled metrics. That’s fine, reasonable people can disagree about what they’re showing – about the signal to noise ratio embedded in these numbers. I’d just like to know what the M’s look for in a catcher. They’ve got two ex-catchers as assistant GMs. They USED to have an ex-catcher as the player development coordinator. It’s not that they don’t believe that framing matters at all – they practice it (when I was ranting about this on twitter,* Ryan Divish noted that he’s *watched* the M’s go through framing drills). Do their internal/proprietary numbers agree with…every public sabermetric study on the subject? Or are they looking for a subtly different skill set, one that isn’t picked up by pitch fx-based studies, but that still affects counts (and runs)? If so, what is it? The M’s have trotted out the worst hitting (miss u Jaso), worst pitch-blocking, worst pitch-framing catchers in baseball, if the publicly-available data is to be trusted.
Several of you responded with theories of how this could be on twitter, and I’d like to hear any others in the comments. Is there a flaw in how they instruct their catchers? Is there some bizarre park effect? Are we fundamentally getting the allocation of blame wrong, and not even Jose Molina could frame a, say, Jason Vargas pitch? Are the M’s catchers systematically undervalued because the run values ascribed to different counts don’t work in Safeco’s run environment? None of these seem remotely satisfactory.
If everyone’s on the wrong track with this stuff, I’d love to hear about it. If there’s *another* hidden-yet-incredibly-important skill that everyone’s missing, that would be a revelation. But I have no idea what that would be. The fact that every single player** has looked so bad (except Jesus Sucre!), the fact that you look up each newly-acquired player they sign and it’s the same horror show…it almost looks like they are targeting poor pitch framers, as if shorting pitch framing was a bizarre market inefficiency.*** If the M’s wanted to punt defense and focus on offense, that would be a philosophically defensible position, but one that running John Jaso out of town at the first opportunity seems inconsistent with. The M’s care deeply about catcher defense. They just care about it differently than everyone else, and essentially opposite to the ways we can measure.
* I wanted to write this up last night, but my laptop wouldn’t cooperate, so I grabbed my phone and chunked much of this into 140-character bites. I’m glad I did, as Divish’s input was important, and it was great to hear folks’ theories. But it does mean some of you are getting kind of a repeat here. I’m still completely baffled by this, if you haven’t noticed.
** This brings us to Mike Zunino, of course. He could be the best framer the M’s have had, but then, an Ivan Rodriguez baseball card, a potted plant or an oatmeal stout could be the best framer the M’s have had. He got good marks on his defense in college, struggled slightly with blocking in the minors, and looked OK in MLB in limited duty last year. I think it’s telling, though, that the one guy who could buck this trend is the one guy unambiguously selected for his bat. It’s when they really focus on defense that they select such poor defenders.
*** As many have noted, the free agent/trade markets do not seem to price framing into player values; baseball as a whole seems not to buy in to the whole concept, which may be telling. That also means you can’t get a “bargain” by picking up someone the new metrics have unfairly slighted.
Comments
20 Responses to “The Mariners and Catcher Defense”
Leave a Reply
You must be logged in to post a comment.
Mmm… oatmeal stout.
Given the decidedly old-school ethos that apparently drives the decision making behind this front office and ownership group, I wouldn’t be surprised if this is simply the fault of relying on the judgement of ex-catchers who really have no idea how to quantify what makes a good catcher, but who “know good catching prospects when we see them”.
I mean… I know we’re not into results driven analysis, but the team has been so consistently bad at building a decent roster overall – why expect competence when it comes to identifying good catching?
I may be drinking an Oakshire espresso stout at this very moment. I love oatmeal and coffee stouts so much, which is odd considering I hate coffee and find oatmeal boring.
Not saying you’re wrong – it’s the best or one of the best theories out there – but why would ex-catchers be so bad at identifying the qualities catchers should have?
We pilloried Mike Scioscia for picking Jeff Mathis over Mike Napoli, and that was *weird* but it turns out Mathis was really good at this, and actually had the cERA that sort of reinforced that framing numbers (not that CERA is a great stat; just as a test if framing runs explanatory power). So there was *something* there, even if Scioscia over-weighted it. With the M’s, it’s ex-catchers scouring the world and bringing back the *worst catchers in baseball.*. Either the numbers are fundamentally (they are literally meaningless, not just overstated) or the M’s are…I don’t really know.
That’s not really the question. The question is more why might the particular ex-catchers the M’s use to evaluate talent be so blind to these particular qualities. Given the incredible consistency of M’s catchers, and M’s general record, over the entire period covered, it just might be that there is in fact a blindness in these individuals. It’s clearly not matched in the rest of baseball, and I’m sure there are plenty of ex-catchers for other teams who are doing a fine job with it (although “guys the M’s aren’t interested in” might be a good way to evaluate catchers defensively without any other information).
As with any other individual statistic, being bad at this one can be overcome. But given that our catchers haven’t exactly done anything else worth writing about (and remember we’ve had a few high draft choices who didn’t make this study because they didn’t make the majors at all), an ownership group that, looking at these statistics, decided to hire someone else to evaluate catchers wouldn’t be making an entirely rash decision.
I have no theories why it might be wrong, other than evidence that the M’s have been pretty good at pitching during that time frame. Are we saying Felix should have been the slam dunk Cy Young winner EVERY season, if we only had league average pitch framing?
Or, that despite our inability to score runs, the M’s would win more games if catchers held their gloves in a different spot?
There’s some whacky causastion/correlation distortion in there. That, and evidently Japanese pitchers thrive with really bad pitch framing? (Sasaki and Iwakuma)
The M’s have been especially good (sans last year) at getting scrap heap/bartender/pot smoker relievers to somehow pitch above league average, year after year (again, last year not so much)… Bad pitchers benefit from bad framing? Like a double negative becomes a positive?
My “gut” says framing is a vital part of catching, but the Mariners might be the team whose RESULTS challenge the data the most. No hitters, especially a TEAM no hitter featuring multiple pitchers, a perennial Cy Young candidate, top of the AL rotation for a couple years…
Perhaps the numbers in the new stats are more like a “temperature” than a run “causer/reducer”– they don’t relate to runs, literally, but still give a reference to measure the skill of the catcher? Maybe just too many other variables (once the ball is in play) overshadow anything the catcher causes during pitches?
There’s also the possibility the “best” catchers are catching the most veteran, established quality pitchers? Who get the “benefit of doubt” calls from umpires? A rookie gets a ball on a pitch in the same exact location that Justin Verlander gets a strike? How do we even begin to account for that? (Adjusted for umpires and pitcher expertise?)
I dunno. It will definitely be an interesting stat to see develop as it gets refined in the coming years.
I’ve never played catcher or anything, but it seems to me that catching a Felix change up would be like trying to catch housefly with your hands.
Sorry, my earlier post was definitely unclear. Breadbaker stated exactly what I was attempting to say. I meant the specific individuals in the evaluation role in this organization – not ex-catchers in general.
One issue could be the calibration of the *data gathering equipment* at Safeco. To test, observe how certain catchers perform before and after arriving at Safeco. Or Home and Away splits of Mariners catchers.
I think MrZDevotee hit the nail on the head here.
breadbaker –
“Given the incredible consistency of M’s catchers, and M’s general record, over the entire period covered, it just might be that there is in fact a blindness in these individuals”
Possible, but let’s remember that “these individuals” have a habit of changing just like the catchers they select do. Pedro Grifol was the ex-catcher in charge of player development, but he was shipped out a few years ago, and a totally different ex-C, Ted Simmons, joined the front office as an assistant GM.
If one ex-C’s spanned all of this, it’s Roger Hansen, and I’ve written before about the, uh, uninspiring track record the M’s had with him as their catching instructor. But he’s *not* the catching instructor anymore. You can argue that Hansen has sold the team on his vision of what a catcher should do and what good catching looks like, but it seems like a stretch. It would also be odd if Zduriencik listened to Hansen’s ideas on this topic to the exclusion of Simmons’. Of course, it’s possible that this small group of ex-C’s *All* had the same ideas of what good catching is, so yes, it’s a distinct possibility.
Sergey –
This is a really important point, but it seems to be one that people from Fast on have dealt with. There very clearly *are* calibration issues with Safeco’s pitch fx system, and it makes the raw data on pitch movement really odd. But Fast came up with a method to correct for it, and Marchi had a similar-but-distinct method of his own. Brooks/Pavlidis are the public face of “corrected” pitch fx data, so I have to think this has been addressed (the article implies that they’re using Mike Fast’s methodology).
Still, it’s something to keep an eye on. One test (since I don’t have home/road splits) is to look at the team switchers – Jaso and Olivo played for different teams in the pitch fx era, while Johnson/Moore/Johjima didn’t. Jaso and Olivo both graded as pretty terrible before/after being M’s, though there’s a ton of variability in Olivo’s line, while Jaso is more consistently-bad.
MrZ –
This is the big question, and it’s one reason why Dave and others have questioned the magnitude of the effect here. If the range is *really +/- 50 runs, which is just gigantic, you’d really think it’d show up in actual runs allowed.
That said, I don’t know if we can say that the M’s have been “pretty good in that time frame.” Looking at 2008-2013, the M’s rank 18th in FIP in baseball despite playing in a HR-suppressing ballpark (the kind of thing that should really, really affect FIP). By ERA, they’re 15th, and by xFIP, 23rd. Felix has been a member of the rotation the entire time, though it’s also true they’ve had some rebuilding years, so they’ve given a chunk of innings to replacement-level guys. Essentially, however you want to slice it, the M’s pitching has been a touch below average, not park adjusted. If you park-adjust, they’ve been below average overall (21st in ERA-, 24th in FIP-).
Notably, the Pavlidis/Brooks study that tries to account for the effect of who’s pitching, not just who’s catching, identifies Felix as having one of the biggest negative impacts on framing of any pitcher. That was in cumulative terms; analyzed as a rate stat, two other familiar names that show up at the bottom are Brandon League and Mark Lowe.
I’d be prepared to consider that some of the bad numbers from Mariner catchers are influenced by the kinds of arms selected for the pitching staff. At the same time, the evidence is strong enough, both statistically and anecdotally, that the catchers have been consistently bad in their own right. As such, it’s completely fair to have serious doubts about the front office’s ability to evaluate catcher defense.
It could be partly that imprecise conclusions add up to make extremes more extreme than they actually are. And if there is inherent imprecision, players should be placed into bins rather than ranked. And finally if Felix or Safeco adds uncertainty, Mariners catchers may be collectively ranked lower than they should be.
That’s not to say Mariners catchers have not been bad, just maybe not as bad as the system pops out.
My impression has been that Mariners management rates game calling and pitcher rapport very highly, to the extent of let everything else fall where it may. And leadership. Catchers strong in these areas may mask other deficiencies that don’t become effectively evident until the highest level of play.
This is the result of poor scouting and player development. Half of those guys they gave time behind the plate never played any significant time for another team after they left so other teams knew they weren’t fit to be catchers in the big leagues. A similar article could be written about the OF defense.
Part of the organizational failure is that the announcing crew for the M’s has nothing to say about advanced stats, nor apparently any understanding. All we ever heard about Miguel Olivo, for instance, was how well he handled the staff and how good he was at blocking pitches, which anyone could see wasn’t true. Just this morning Shannon Drayer said something like “Zunino’s defense is normally strong, but he struggled last season”, which to me is a contradictory statement.
That makes me wonder about Dan Wilson. Is there a stat that can measure how good Wilson’s defense actually was? That’s all we heard about for a decade, but much of that same decade was characterized by some truly lousy pitching seasons, particularly out of the bullpen. I’ve never been able to reconcile the consistently terrible bullpens with Wilson being a good catcher. Is there a stat we could apply to Wilson’s catching that would help us?
I wonder if the M’s simply recognize they wouldn’t/wont contend and refuse to stress catching value, much like it appears they don’t with first base duties and, more recently, outfield defense?
The Mariners have a proprietary metric to measure the hardness of a player’s nose. Rob Johnson was off-the-charts on hard-nosedness. Olivo too–that guy could hammer nails with his nose.
Oh, how I miss the days of Bob ‘Scrap Iron’ Stinson….He probably could still hold his own in relation to some of the M’s mentioned in this piece…..In fact, that would be a fun article–go back to the beginning and see how M’s catchers B.W. and A.W. (before Dan Wilson and after Dan Wilson) compare to each other….