On That Which Does Exist
By way of Jeff Evans, this is an article by Colin O’Keefe that examines the state of the Seattle Mariners’ analytics department. While, by very definition, the lead isn’t buried, I’d say that it is somewhat glossed over: the Seattle Mariners have an analytics department. That is a thing which exists, and I recommend you take the few minutes to read the article, if only for the emotional comfort of seeing the Mariners linked to HITf/x. It’s right there on the Internet, and the Internet hasn’t lied to me since that one time it said the Mariners walked a Padres hitter on three balls. Haha, Internet, you’re so crazy.
It’s a good article based around a good conversation, and it undermines the caricature of the Mariners, where they’re a hopeless organization that has everything completely backwards. The team, obviously, has bright and motivated young people working for it. The goal is to win, to be sustainably successful, and these people wouldn’t get paychecks if they didn’t serve some purpose. I guess they could be unpaid interns since baseball is a miserable industry to work in when you’re just starting, but then replace “paychecks” with “responsibilities”. The Mariners’ front office isn’t a bunch of 60-year-olds farting around a conference table.
The rest of this post isn’t intended to come off as negative, so try not to read it that way. Rather, I just want to write out a few follow-up points after reading O’Keefe’s article twice. He had a good chat with a smart guy who works for the Mariners. If he blogged about a different team, he could’ve had a good chat with different smart guys who work for other organizations. The reality of baseball today is that just about every organization has young thinker-types, working on numbers and trying to find value. The Phillies might be the last holdout, or possibly it’s the Twins, but even the Phillies are beginning to crack. Analytics is just a part of the game now. Fans tend to exaggerate the mindsets of the unsuccessful. Basically every team is using computers, and while it’s encouraging to know the Mariners have a whole staff working on the smart stuff, that serves to answer only the first question. As I see it, there’s more to explore.
(1) Are the analysts good?
I don’t mean good, in a vacuum. I mean good, relatively. You could go hang out with Pete Kozma. You could spend a whole week with him, training and taking grounders and batting practice. You could talk to Kozma about his approach and his thought processes, why he does things the way he does, adjustments he’d like to make. You’d come away pretty convinced that Kozma is an awesome player, and he is an awesome player. He’s an amazing shortstop. He’s also a profoundly mediocre shortstop by big-league standards, and he’s a shortstop the Cardinals are going to try desperately to improve on in the coming months. Kozma doesn’t get points for being better than most baseball players. He loses points for being worse than his big-league peers, and this is where we don’t know enough about the Mariners. And can’t know, really. Especially since they’ll never go into detail explaining their own methods. How good is the Mariners’ analytical staff, relative to, say, the Angels’? The Rangers’? Dare I ask, the A’s or the Astros’?
These days, an analytical staff is basically like a shortstop. You’re just expected to have one and you’d look silly if you didn’t. All right, everybody has shortstops. How good is your shortstop? Do the Mariners have Troy Tulowitzki? Elvis Andrus? Daniel Descalso? I’m not asking as a literary technique as I get to the answer. I don’t have the answer. Doubt the Mariners do, either. Don’t think the analysts are spending their time analyzing the analysts. That’s a little too meta for the real world.
(2) Do the analysts matter?
We know that the department serves a purpose, because if it didn’t, it wouldn’t exist. When Tony Blengino stopped serving a purpose, he disappeared after an awkward stretch. The way it’s laid out is that the department is free to explore its own ideas, and sometimes specific requests are filed, and the information generated becomes part of the conversation at the highest levels. But what’s the significance of that information to Jack Zduriencik and his closest associates? Can it drive decisions? Can it tip them? Is it almost entirely ignored, like it’s been for some time in Washington, at least in years past?
Zduriencik has said several times in the past that everything is a group decision, and everybody gets a chance to express his or her own feelings. Ultimately, it falls to Zduriencik to make the calls, so he decides how much he cares about what the numbers say. We have evidence to suggest he listens to the numbers less than he used to, but that isn’t scientific and, additionally, minds can change. Zduriencik’s mind has changed at least once. It’s possible the Mariners have an awesome analytics department that gets ignored. It’s possible they have a lousy analytics department that doesn’t get ignored, which might be worse. Everything in between is possible, too. Every team has to decide how much it’s going to trust its own eyes.
(3) How much trickles from the top down?
That is, while the analytics department could be very talented, within what kinds of framework does it go about its business? What’s the role if, say, the Mariners decide they need to field a winner in 2014? What if the Mariners decide to change focus or shift priorities, like from defense to dingers? Does the department try to come up with the best suggestions that fit a theme, or does it try to come up with the best suggestions, just? To what extent is it a two-way relationship between the analysts and the general manager? I’m not doing a very good job of exploring this point, but I can’t think of what I want to say, exactly, so hopefully this at least touches on it.
There’s also the matter of the elephant in the room — how much do we overrate the importance of analysts within a front office? How much do we overrate the importance of “sabermetric thinking”? There aren’t that many disagreements over who is and isn’t good. This year, the Cardinals made the World Series, and their strength isn’t exactly sabermetrics. The Tigers were an awesome team all season and they’ve never been thought of as progressive. The Red Sox won it all, and of course the Red Sox are famously sabermetric, but then for one thing, this year they put so much emphasis on chemistry, and for another, the 2013 Red Sox are no more or less representative of the front office than the disastrous 2012 Red Sox. There are so many other things that contribute to winning, and there’s so much random variation, and we still don’t properly appreciate the Tampa Bay miracle. 90+ wins in five of six years. 84 in the other. Miracle. We like to think the Rays validate the sabermetric ideas, but really, the Rays validate unparalleled genius. No one should realistically be held to that standard.
I think the fact of the matter is that while we want the Mariners to be smart, and while the Mariners should want the Mariners to be smart, the key for them is probably going to be player development more than anything else. Things would look super different if the team could count on bigger things from Dustin Ackley, Justin Smoak, and Jesus Montero. What if Danny Hultzen were healthy, and Brandon Maurer were better, and Hector Noesi didn’t suck a lot? Obviously not everyone can be a success, but strong internal development is what you should count on to make a mid-market team sustainably competitive.
Analytics as we understand it is mostly about players in the bigs, or the very high minors. The goal is to identify talent and value, and just about every team has a department trying to do the same thing. Similar departments will identify similar players, and then there are only so many to go around. You can’t acquire all the value you identify, and your exclusive talent pipeline is your own minor-league system. Those are the players that only you have, and those are the players who can make and keep a team good for a while. The Cardinals’ player-development system seems awesome, and the Cardinals are always good. Same goes for the Rays. The Pirates’ player-development system sucked when the Pirates sucked. They couldn’t dig themselves out until they started churning out their own talent. They’ve made some shrewd moves in the market, and they’ll try to make some more, but right in the middle you’ve got McCutchen, Marte, Alvarez, Walker, Cole, Morton, and Locke.
And you’ve got whatever made Francisco Liriano good again. Sabermetrics didn’t chop his FIP by a run and a half. There’s player development, and there’s player rehabilitation, I suppose. Maybe a better umbrella term would be talent maximization. That’s where the Mariners really need to be better. Some of that, probably, is analytics, but it’s also scouting and coaching and ensuring general wellness. You know, the kind of stuff we can’t speak to. We love analytics because at least we mostly understand them.
The Mariners have an analytical department, staffed by multiple people. Those people take requests from the bosses above, and they’re involved in decision-making. They are, probably, quite smart, and quite driven. We don’t know how good they are, relatively speaking. We don’t know how significant their role is within the Mariners organization. We don’t know how much they’re driving decisions, and how much they’re responding to ideas. And at the end of the day, numbers aren’t going to make young talent better, not by themselves. I’m pleased to know at least one person within the Mariners organization spends time with HITf/x. I’d like to know if Justin Smoak’s ever going to slug .450.
We may not know too much, but we can make an educated guess that, while the analytics guys are probably consulted on decisions, they don’t drive most of the decisions. I’m pretty sure they didn’t make the decision to start the year with 6 DH’s, for instance. The A’s give the impression of a team that uses analytics to start the discussion on any decisions they make; the Mariners just don’t seem to work that way.
I went to a game against Tampa Bay this year, and it was fascinating to watch their infielders moving all over the place, switching positions and playing a different alignment on different counts. A woman behind me asked the guy she was with how to score a play, and I realized I did not know. The ball was hit to Evan Longoria, nominally the third baseman, but in this case he was playing on the first base side of second, with the shortstop still more or less in his usual position and the “second baseman” in short right field.
I don’t really care how you score that play, whether it’s 4-3, 5-3, 6-3, or 4.5-3, but I want a team like that. Old school isn’t working. They need to try getting out in front of the league instead of lagging behind.
Jeff, it seems to me — to me, I can’t stress that enough — in your last few posts, that you are saying “we don’t know” a lot. And maybe you think we aren’t keeping that in mind: Basically, we don’t know, about everything important. And you’re right. We don’t know. It makes reading a baseball blog just, well, chatter.
Even though you’re basically right, it makes me despondent. Even more than thinking about the Mariners.
Fortunately, we also have Dave to balance out some of Jeff’s uncertainty.
I don’t know a way around not knowing enough. I’m more comfortable just saying that, acknowledging it, than trying for conclusions, but I am aware that a lot of my posts end up feeling distinctly uncertain. It comes and goes as different ideas come and go. With this one, I think uncertainty’s appropriate, although gun to my head, I’d offer a few ideas about how I think the Mariners really use their analytics department.
Gun to my head, I’d say “sparingly,” “not enough.”
Regarding the importance of sabermetric thinking…What the A’s and Rays do is nothing short of astounding. In what should be a highly competitive industry with the best minds going against each other, they stand far, far above most of their rivals. Whether you want to call their methods “sabermetric thinking” or something else, I don’t see how you can be a baseball owner or executive and not want to jump on that train as fast as possible.
And if you really don’t have the front-office talent to compete with those teams, you need to keep looking until you find better front-office talent.
Consider the Jaso for Morse trade. Did anyone in the analytics department shout at the top of his lungs, “STOP!”? We don’t know.
Justin Smoak can slug .450. He’s done it, just not for extended periods. With Smoak, I think he needs to work on his bottom more than his top, limit the bad stretches. If in 2013, Smoak had an extra single for every 17.3 ABs, he would have slugged .456 for the year. 20 more singles and a .282 BA and he’s a >.450 slugger.
“He had a good chat with a smart guy who works for the Mariners. If he blogged about a different team, he could’ve had a good chat with different smart guys who work for other organizations”
Here’s the accompanying visualization, found doing research for the piece on LinkedIn: http://bit.ly/1hULMcF
Yep, there’s an armada of these guys. And we have no idea what their ultimate impact is.
Analyse this:
Howard Lincoln – Hot Dog Vendor.
It’s certainly true that teams that seem to use a lot of analytics can be good or bad, and those who don’t can also.
If we really knew how much each team made their decisions based on the data, would we find a strong correlation between that use and the number of wins for the team?
Here’s my main question, and my main problem with depending very heavily on sabermetrics.
The analysis is of the past–events that have already happened. Yet the decisions made are for future events. Has anyone ever done a study of the predictive accuracy or value of sabermetrics? When I read that one stat was an abberation and another should regress toward the mean and so the player should do such and such next year, has anyone ever studied statistically how accurate that is?
Dennisss,
100% correct.
Jeff, I enjoy reading what you’ve written – even when you’re uncertain about everything. 😀
It sure would be nice to have access to that Hit f/x data…
The underlying article is interesting as is your take on it, Jeff. This suggests Jack Z is who we thought he was when he was hired, though the loading up of DH’s last year will always be a puzzle. The difficulty of projecting performance, even using all tools available also suggests to me that my enthusiasm for Jack Z based on his draft successes with the Brewers was misguided and underestimated the amount of luck involved. Rather than him suddenly becoming a dunce or luddite, perhaps a better explanation is regression to the mean in terms of luck of projections–maybe even beyond the mean to a decidedly back luck streak. We don’t have the money to offset a lot of bad luck, so here we are, but maybe that luck will change? That plus spending more money to allow some better gambles with more major league proven talent…
I wonder if the team’s analytic department ever butts heads with the team’s psychic department.
^That’s funny. That could explain why they chased HR’s last season.
I don’t think we can blow off Jack’s last off season quite so easily. Was there anyone here that doubted the team had an analytics department, prior to the publication of the LL article? It’s not like these guys weren’t already with the team 12 months ago.
I don’t get why suddenly people are saying “hey maybe Jack isn’t so bad…” – nothing substantive has actually changed from a week ago or a month ago. Now we know the names of some guys who were already employed by the Mariners prior to the Bay/Morse/Ibañez/Morales signings, and we didn’t know their names previously. That’s pretty much it.
I’m with Westy-
All that article told me is that maybe we need to replace our analytics guys too? I mean, if we’ve had a true analytics department (rhetorical) and have been as awful as we have at finding guys of value, other than Jaso, then they’re part of the problem too, aren’t they?
That said, it was a cool article to read… I finished with the impression that the analytics guys are the baseball equivalent of archaelogists doing excavations all over the place, trying to uncover the next great discovery, and cash in on it. With a bunch of other archaelogists out there trying to do the same thing. Bottomline it feels like if you’re gonna run one of these departments, you really need to have the VERY best guys at it, because everyone else is in on it too.
It would be interesting to hear what sort of analysis the Seattle department offered up on moves like Jason Bay, Ibanez, Chavez, etc.
I’m pretty sure that the best statistical analysts work at hedge funds and get paid more than baseball players, let alone baseball analysts. So unless baseball teams start upping the pay rate dramatically, they just won’t be able to compete for the top talent.
Does anyone know the status of Endy Chavez? He seemed to simply vanish last season. Did he injure himself? Was he traded? dismissed? did he end up going to the Japan league? I thought he was one of our most solid performers last summer.
In 2013, Endy Chavez posted a below league average on base percentage while hitting for less power than Brendan Ryan.
Go look it up.
I like Endy, but he’s done as a useful player. I know the announcers loved him, but seriously – go look up even his basic numbers. He wasn’t “solid” last season.
I enjoyed reading it all, although Jeff, I offer one edit to your writing:
“The Mariners front office isn’t ONLY a bunch of 60-year-olds farting around a conference table.”
Because there are certainly some of those there as well.