Evaluating Managers
I rank a manager’s jobs in importance to remaining employed as:
– Keep his team happy and motivated or, barring that, motivated
– Deal with the press
– Pick the players for each day’s lineup
– Manage the pitching staff
– In-game tactics
– Team strategy
– Prepare his team for each series and each game
– Arrange the day’s lineup
This is not how important they are to his team’s success. A manager could be abjectly horrible in post-game press conferences, cold to reporters in general and terrible in his call-in radio show, and it wouldn’t cost the team a run. But it would affect perception of him tremendously, and the baying dogs would chase him night and day.
It could cause more than a run’s damage, too. You can see where if a manager was constantly under fire from the media (and we can safely assume, the fans, because he’d come off terribly on TV), it might spill over to the coaches and players and cause them pain. So even the seemingly trivial skills tie into the first and most important: keeping the team playing. If a manager’s slagging his team after every game, it might motivate them, but it’s much more likely to piss them off and make them not want to give their best effort, our of spite and because doing so would make the manager look good.
Bobby Cox is great at keeping his mouth shut: it’s hard to come up with an instance where he’s said anything bad about any of his players. It makes him a little boring, but if you’re a player, I can understand that you want him to chew you out for a mental error in private and protecting you in public.
But again, we don’t know the difference in actual team performance between a Cox-lead team and, say, a team led by a frustrated Lou Piniella.
The value of keeping a team happy and motivated is hard to objectively measure. Do the players complain about the team or praise it? Does that really have more to do with players or the manager? At the extremes, this is noticeable: players generally love being on Dusty Baker’s teams, even as he gets them to play bench roles or platoons. Some managers, who don’t last long, can’t get the respect of their players. But in between, it’s quite hard to tell the difference betweek the good and that below-average.
Trying to guage it by performance is tough. Almost every team plays horribly for some stretch of the season. They bobble balls, make stupid baserunning mistakes, chase bad pitches every at-bat. And even the worst teams put together a week or two where they make fans wonder where they’ve been all season. There are some danger signs a fan can spot. But do the players consistently seem to space out, or give half-assed efforts defensively? Do they bitch about their playing time and the team’s direction? Do plays break down when signs are missed or misunderstood?
On the whole, though, we’re a long way from being able to measure the ability of managers to motivate their players.
Let me try that list again in terms of things we can attempt to measure:
– Pick the players for each day’s lineup
– Manage the pitching staff
– In-game tactics
– Team strategy
– Arrange the day’s lineup
Daily lineups
Picking the players who take the field is the most important part of a manager’s job. A manager has great leeway on how to put together each day’s team, and they get to use it 162 times a season.
This is also where it’s possible to evaluate a manager’s preferences and do quality second-guessing.
For instance, say the team has a young corner outfielder who plays good defense and hits like crazy against both left and right-handed pitchers. But the manager insists on platooning him with a weaker, scrappy hitter he likes.
There are two possibilities: one is that the manager sees something flawed in the kid’s approach, and is covering it. The other is that he’s making a mistake, and valuing insubstantial factors over actual performance. This happens all the time with managers picking veterans over minor-leaguers, even minor-leaguers with amazing track records.
There are sometimes considerations that outweigh baseball reasons. If the team wants a manager to showcase a player for a trade, the manager might play them in a strange position, or start a horrible pitcher in the hopes that he’ll string together two good games and convince someone to bite.
But by and large, how does the manager distribute his playing time? Does he platoon and find ways to allow his players to succeed, or does he somehow get the crappy utility guy 250 ABs instead of 110? Is he able to find ways to rest his regulars without hurting the team’s defense or offense too badly? If a regular player goes down, how does he patch the lineup for the week, and how does he adjust the lineup?
Manage the pitching staff
The most obvious criteria is whether they regularly overuse pitchers. To distill the years of debate on this, throwing over 120 pitches a start is bad for the pitcher’s next start and their overall health.
Baseball’s been remarkably civilized about this in recent years. Most pitchers never go over 120 in the course of a season, and the per-game average for even the most heavily-used pitchers is around 110.
Less obvious is the “quick hook” and “too late to pull” qualities. If a pitcher’s clearly tired and throwing poorly, does the manager get them out early, or do they allow them to keep throwing and lose the game? Conversely, are they so aggressive that they remove good pitchers after brief struggles? It’s hard for us to easily measure this. We can try to look at quality starts and blown quality starts, but those are imperfect measures.
As with picking the lineup, there’s also some effect in setting pitchers up for success, and protecting them from failure. While generally a manager’s going to be better off with the pitchers on hand compared to trying someone new, there are instances in a season where there are chances worth taking. For instance, it can be worth pitching a left-hander against a team even if that means calling up a random AAA guy and re-working the rotation if the opponent is so weak against left-handers. A good manager constantly looks for chances to make adjustments that let him steal those advantages, and a poor one does not.
The bullpen management provides many way to evaluate a manager.
I reject, immediately, the notion that a team needs a 9th-inning-only closer, and that closer needs an 8th-inning setup man. This is a dead end in baseball’s evolution, and eventually we’ll get past it. If a manager displays flexibility and insight in playing matchups over roles, especially if they’re willing to call in their best pitcher when they most need them, whether or not it’s the ninth and they’re up by 1-3 runs, that’s a great thing. Few managers are that smart.
In general, how does the manager use his relievers? Does he recognize talent, and use it when it’s important, or does he keep ineffective veterans
In a more subtle sense, does the manager seem to take the strengths and weaknesses of his pitchers into account? If a reliever is overall quite good but has a home run problem, and there are two runners on with a slugger up, up by two on the road, you’re better off bringing in a less-effective groundball pitcher. In individual instances, good moves will backfire and bad moves will work out sometimes, but if you continually see the manager bring in a homer-prone reliever in situations where a home run could cost the team a game, that’s a good indication he’s not being smart. You should be especially worried if their justification for a particular move displays a remarkable ignorance of the facts — say, they bring in a homer-prone flyballer instead of the groundballer, and then claim that the flyballer was the best bet to get a ground ball double play to get out of the inning — because if they don’t know their staff’s talents and weaknesses, they’re not going to do well trying to play to them.
In both the bullpen and the rotation, are they willing to make changes when changes are required, or will they stick with a player in a role they’re unsuited for long after it’s obvious to everyone else?
In-game tactics
First, the running game, because this is something we can easily set a few metrics for. Most statistical research will tell you that a stolen base needs to work 75% of the time to be effective. There are situations where one run is important enough that the reward makes the extra base worth more, and it’s worth trying a steal if you think there’s only a two-thirds chance they’ll make it. You can also argue the threat of the steal has value, so it’s worth a few counter-productive steals to establish that.
But really, anything less than a 67% success rate means the manager, in total, is taking runs off the board and hurting his team. As I write this, the success rate of teams ranges from 80% (Mets) to 59% (Braves). The Mets advanced 90 more bases without creating an extra out. That’s a huge swing in the team’s fortunes over the course of a year, and easily two games in the standings.
So there are two things to look for:
– The team’s total success rate at stealing bases and number of attempts
– Who’s stealing
The Braves sucked at stealing bases, but they also were close to the back of the pack in total attempts. While we could argue that they should have taken it even farther back, they weren’t running themselves ragged counter-productively.
Meanwhile, the Mets were among the best and took advantage of their strength. The Phillies had a great success rate and didn’t, for reasons I’ll get into. We can say though that we know that the Mets did a good job with their running game, and though the Braves weren’t, they at least didn’t try and push it, as the Nationals and Marlins did with only slightly (66%) better success.
For the second, who’s stealing, let’s take a look at the Phillies and Mets, both of which had good years on the bases.
The Phillies have one runner, Rollins, who is 36-4, Abreu was 20-4, Chase Utley is 14-4, and Aaron Rowand is 10-4. No other player has more than 5 attempts.
The Phillies are having most of the team sit still, the guys with good speed taking good chances, and Rollins has a green light to steal at will. That’s an excellent strategy: as we don’t want teams to steal for a low success rate, it only makes sense for them to steal with players that can do it well enough to swipe successfully at least two-thirds of the time.
The Mets are nearly the same way: Reyes is the speedster, Wright and Beltran have 25 and 20 attempts, and Endy Chavez has 15. They have more players who’ve gone under ten times, but you see nearly the same pattern: they’re playing station-to-station with the slow guys and letting the faster ones win.
This seems like an obvious strategy, but it’s not at all. Many teams, for whatever reason, keep stealing with guys who get caught. Florida’s Alfredo Amezaga has 19 SB that cost the team 12 CS. Jamey Carroll has 10 SB to 12 CS.
Next, for another quick measure of a manager’s brain activity, look to the team’s count of sacrifice bunts. The sacrifice bunt is almost never a good move. The more a manager uses it, the worse they are. The more they can resist the temptation to manage by the book, the better off the team is.
And it’s also a situation where managers vary enormously. This year, Colorado was credited with 116 sacrifice bunts. Toronto made only 16. Colorado sacrificed over seven times as often. That’s enormous. Of course Colorado’s an NL team, so that’s an unfair comparison, because NL teams use their pitchers to sacrifice all the time. As you’d expect, the top 16 teams are all NL teams, and the bottom 14 AL, but it’s not as stark a division as you’d think. Here are the ranges as I write this:
AL Teams: 16 (Toronto) – 50 (Kansas City)
NL Teams: 56 (Philadelphia) – 116 (Colorado)
KC was 2.5 times as likely to sacrifice as Toronto, and Colorado 2 times as likely to sacrifice as Philadelphia.
Managing the Phillies, if you’re curious, Charlie Manuel, formerly of the Indians.
There are other subjective measures a fan can look at:
– when are they stealing?
– are the sacrificing in really stupid situations where giving an out is dumb, or are they relatively smart choices?
Beyond that, it’s hard for a fan to make evaluations of a manager’s in-game strategies. Too much of it has so many factors that go into a decision that it’s hard to call any single decision wrong, or right. You can still try, though, to answer another big question:”Does the manager make effective use of his bench during games?” Or, too see this another way, does the manager find ways that the bench players can get into games in a way that allows them to contribute?
If he has a player who is a clear liability defensively, does he find a defensive caddy for them, and make use of them? If there’s a guy on the bench who can’t play much defense but is a huge left-handed power hitter, does the manager seize chances to send him in to pinch-hit? Or does everyone on the bench rot?
Even if you hold back criticizing particular instances of pinch-hitting or substitution, you can still get a sense of the manager’s priorities and beliefs that allow you to make judgments about those beliefs.
For instance, say a manager never pinch-hits for his catcher, wishing always to have two catchers available (one in the game, one on the bench). As a philosophy, this is clearly risk-averse and can cost his team runs: if the backup catcher is a .150 hitter and it’s an extra-inning game with runners on where he has some quality pinch-hitting options, the risk is worth it.
And so on. You may also notice if a manager is particularly inflexible about rules, which may not only be bad because the rule is dumb but made worse because unwavering adherence to those rules means that other managers can take maximum advantage of them.
Team strategies
Do they run the bases aggressively, or conservatively? Should pitchers make the fastball their primary pitch and throw it at least 70% of the time?
These are extremely hard for fans to judge. We’re reliant, generally speaking, on people who have access to play-by-play databases and pitch distribution information to crunch these kind of numbers for us.
Arrange the day’s lineup
I get a little annoying about criticizing lineups because I think teams should relentlessly chase every advantage, trying to win every game any way they can, but lineup simulations and discussion around this has generally agreed that it’s not that big a deal. The effect of having Joe Scrub playing anywhere instead of a good player is much greater than the difference between an ideally ordered lineup and a badly ordered one.
That said, it’s still an area where the manager controls his own destiny, and really should try and get what he can. There are many considerations into where each player goes.
Baseball-wise
– L/R handed (force the other team to burn relievers if they seek situational advantages)
– Hitting skill profile (contact/power/pitches seen/etc)
– Special hitting abilities (can the player bunt, or steal)
For example, one of the problems with the Olerud/Martinez lineups is that while both of those guys were offensive threats, they could not run the bases worth beans. They weren’t good at advancing on hits, they were easy to double-up at second, and once on, they clogged the basepaths for players behind them. You can’t go first-to-third on a single if a slow guy is barely going to manage to get second-to-third on the same hit.
It’s an interesting dilemma. The chances that the base clogging hurts the team during the course of a game are pretty slow, but it will happen, and then you’ve hurt the team. Wherever you put them in the order, putting speedy runners after them will negate part of the value of their speed, and putting ground-ball hitters behind them will dramatically increase the GIDP rate.
Intangible-wise
– traditional fits of players to lineup roles (#1 is highest OBP, #2 high contact+bunting, #3 best hitter…)
– player mental fit (does the player try to hit to all fields when hitting #2 but not #6)
This is where a lesser manager says “screw it” and slots his guys into the traditional order their skills suggest. If that means a .245 hitter with great speed leads off, well, so be it. Good managers will come up with lineups that make the most of unconventional talents. We’ve seen this on some of the A’s teams, where devoid of speed they manage to still put together an order where good baserunners with high OBPs bat early.
For a manager, taking the easy path means that they don’t have to spend a lot of time being hassled about it in the press, because any deviation from the most conventional lineup means they’ll be at least questioned about it at length if the team loses. But if they go with the most conventional arrangement and they lose, it’ll go without comment.
Lineup arrangement is a strange art. It’s extremely complicated, with many factors, some of which we can’t know from the outside. But that doesn’t mean that we can’t think about it, and compare our work to the managers, and it certainly doesn’t mean that they’re doing it right.
What can we know?
At a high level, we can evaluate easily only the running game and use of the sacrifice bunt. They’re readily available, sortable to compare against other managers, and useful. We can track pitcher workloads easily.
If you’re willing to spend time on it, who gets playing time and how the relievers are managed are crackable, though it’s almost impossible to do that for every manager, every game, to see how
But unfortunately, so much of the information we need to make comparisons is proprietary, and there aren’t readily usable tools for fans to use. It’s really a shame.
Hargrove sucks
It’s true.
Update: for some interesting data, check out Chris Jaffee’s study: 1, 2, 3. I’m not sure why the links don’t work on their own — thank goodness for the Wayback Machine.