Fan Scouting Report
Friend of USSM and extremely smart guy Tom Tango is putting on his annual Fan Scouting Report. With all the work done to try to quantify defensive analysis, Tango is taking the Wisdom of Crowds approach. None of us are defensive experts, but if you’ve read the wisdom of crowds thesis, you realize that we all have our own small insights to add, and when you get a large representative sample, you can end up with some very good information.
By bringing together a lot of people who watch a lot of baseball, he’s compiled ratings based on the popular consensus of how good defensively players are. The data certainly isn’t the be-all, end-all of defensive analysis, but it’s very good information to have.
So, if you’d like to participate, go fill out the survey. However, please read the instructions. Tom takes a unique approach to evaluating defensive skills, and one that I whole-heartedly agree with; he wants to know what kind of skills a player has, and not how those skills compare to the relative merits of the other players who man that position. Or, as he states it:
Try to judge “average” not as an average player at that position, but an average player at any position. If you think that Chone Figgins has an average arm, then mark him as average, regardless if you’ve seen him play CF, 1B, 2B, or 3B.
DO NOT CONSIDER THE POSITION THE PLAYER PLAYS!
DO NOT CONSIDER THE POSITION THE PLAYER PLAYS!
DO NOT CONSIDER THE POSITION THE PLAYER PLAYS!
Position-independant defensive analysis is superior, in my opinion. Do we want to slag Derek Jeter’s defense at shortstop because his range isn’t good, relatively, to other shortstops, while giving Troy Glaus a pass because his defense at third isn’t as bad compared to other 3B? No – Jeter’s a significantly better defensive player than Glaus, but that gets lost in the shuffle of defensive position rankings.
So, let me encourage you guys to go fill out the survey. Tom’s been extremely helpful to us, and this is a great chance for us to return the favor.
I must be missing something — wouldn’t that method wind up with (just as examples): Nearly all of the catchers lumped together as slow, but with good arms? Most of the 1B lumped together as relatively immobile, and usually with poor arms? You’d think that method would tend to show what most of us would view as stereotypes from each position.
My one criticism of Tom’s system is that it doesn’t take path to ball and geographical presence on the field into account as its own category. Not to taint the report, but Bloomquist is a good fielder once he gets to the ball, but his lack of instinct as to what the ball is going to do turns routine plays into Rick Rizzs orgasms because he’s desperately trying to compensate for his poor path to the ball. Beltre is a good fielder, but it seems like he does not have any idea where he his, where the other fielders are, and where the runners are, so he can be easily distracted. It’s as if he’s trying to get the ball and damn every other piece of information out there. By geographical presence I mean knowing where you are on the field and where everyone and everything germane to that play are.
It might be helpful to go look at last year’s results. That might answer some of your questions.
But basically, no.
Hey Dave, I love the website and all of the great pieces like this one, but where do your ideas come from. Either you have a very intelluctual baseball mind or you need to get laid. 🙂
I must be missing something — wouldn’t that method wind up with (just as examples): Nearly all of the catchers lumped together as slow, but with good arms? Most of the 1B lumped together as relatively immobile, and usually with poor arms? You’d think that method would tend to show what most of us would view as stereotypes from each position.
Perhaps the average fan doesn’t have the stereotypes you expect.
I finished the Mariners’ ballot. Looks interesting. Thanks for pointing this out!
I have a hard time trusting the average fan when they all were telling me that David Bell deserved a Gold Glove for his defense at 3rd base while with the Mariners.
His lack of range continually had him turning a routine groundball into a “spectacular diving stop”, so people always thought he was great because he could making a diving stop and get a guy out.
I have a hard time trusting the average fan when they all were telling me that David Bell deserved a Gold Glove for his defense at 3rd base while with the Mariners.
David Bell consistently ranks as one of the top few defensive third baseman in the game statistically. The scouting reports, both from fans and real scouts, agree. And, it’s very common for a player who moves from an up-the-middle position to an easier spot in the defensive spectrum to outperform his peers.
So, pretty much every piece of evidence points to David Bell being a terrific defensive third baseman. You might want to reevaluate your position.
I agree with frenchonion. (Did you consider changing your name to freedomonion in light of the controversial French-US relations of the last few years?)
A consequence of Tango’s preferred system is that the best athletes will rank near the top.
Another issue is judging arm strength. Ichiro makes long throws, 3b throws to 1b and Catcher throws to second base. How is a fan going to adjust for the differences in required length of throws for determining arm strength?
But Tango is the man so it’s hard to argue with his approach.
A consequence of Tango’s preferred system is that the best athletes will rank near the top.
If the best athletes are the best defenders (which is generally true), then I don’t see how this is a problem.
If the best athletes are the best defenders (which is generally true), then I don’t see how this is a problem.
Because then it becomes a rank of athletic ability, and less of a rank of baseball ability. And that’s fine if that is purpose of the research.
I remember seeing clips of Michael Jordan on the field. Man, he looked clumsy!
Thanks for the support Dave.
To answer the positional-aspect, since I get this question asked more than any other combined:
1 – if you are a real scout, and you see Betancourt in Cuba, you don’t give him a “70” for “3b arm”, “75” for “ss arm”, “80” for “2b arm”, etc, etc. Every player is simply evaluated based on his skillset. That evaluation drives where he should play.
2 – everyone has their own bias as to what’s “average hands” for 2b. If I take you out of the position-mindset, and simply say, “listen, you’ve got 15 guys on your team, and you’ve watched 150 guys on your team in your life, and you’ve watched hundreds of others on other teams… does WFB have average hands, above, or below?”
3 – Sure, as it will turn out, the score for the average 3B arm will be high. For example, this was 2003:
http://www.tangotiger.net/scouting/scout_Results3B.html
The line at the bottom gives you the average score for each trait. And, as expected, the arm strength for the average 3B is “65”, which is very high, and a speed of “45” which is below average. So, good job to the fans, for being able to take the guy out of the position.
4 – Even if the position-to-position comparison won’t be valid because the fans didn’t do enough to take them out of their context, the intra-position comps will still work. Knowing that the average 3B arm strength is “65”, that gets baselined, and every 3B arm above that is above average.
5 – As for David Bell, here are his scores for the last 3 years:
Season Inst. Accel. Speed Hands Release Stren. Accuracy
2002/03 66 56 33 71 62 54 59
2004 61 51 32 52 60 52 55
2005 47 43 34 48 54 46 62
Not sure how well this will format. The 02-03 season was based on Giants fans, and the other two by Phillies fans. All had around 15 or so ballots.
As you can see, the fans see him as getting a bit worse each year. His instincts, acceleration, hands, release, arm strength all getting worse. His speed remaining the same slow speed. And his accurracy staying around the same.
(Of course, the smaller the number of ballots, the more random variation we have to account for, which is why we’d like as many hardcore fans as possible.)
I have MGL’s updated 2000-2005 UZR, and he does quite well. In his 2000-01 Seattle seasons, he was above average (about +10 per 162 GP). In 2002/SF, he was great (+30 per 162). In Philly, +15 per 162. So, over that whole time period, he was right around +16.
I would say that the way the Fans responded, they see him as a below average 3B, especially in his later years. His UZR however continues to rank him high. If you have an issue with the way Bell is perceived, it’s not with the fans, but with the numbers.
This is the point of the survey, to actually figure out what it is that fans see, and we can keep that as a(n) historical record. This is not to say they are right, but at least, we have a basis for discussion.
Because then it becomes a rank of athletic ability, and less of a rank of baseball ability. And that’s fine if that is purpose of the research.
I remember seeing clips of Michael Jordan on the field. Man, he looked clumsy!
They’re evaluating their athletic abilty based on how they play baseball. A fan watching Michael Jordan being clumsy on the field would rank him lower for that reason — and thus your example helps to defeat your argument. Michael Jordan doesn’t get extra credit because his athletic ability enables him to dunk; Michael Jordan gets penalized because his athletic ability isn’t sufficient to overcome his lack of baseball skills. You have to be able to apply your athletic abilty to the game of baseball, and fans seeing you do so will rate you higher. Which is exactly the point of this survey.
tangotiger,
Thanks for the response.
I see what you’re doing now.
But I disagree that the fans are getting it right. (I include myself in that statement.) For example, Ichiro is the best RF and he is rated as having a stronger arm than the best 3b, Scott Rolen. How can that be? A RF, any outfielder for that matter, usually gets a nice run in which he can put his body behind the throw. The 3b doesn’t have that luxury. That does not show that fans are able to make sense of what they’re seeing.
I’d like to be to find an absolute ranking yet I don’t know that we’re getting it right.
joser,
You are right.
Is it pronounced with the Spanish j or the Am/English j is in java?
Right, they may not be getting it right, but it’s the best we’ve got, given our resource and logistic constraints.
Back in my younger days, they used to do “skills competition”, and everyone was in the same boat. Everyone would bunt, steal 2b, throw to the cutoff guy, etc.
I know it’s asking alot to the fan to say “how strong is Furcal’s arm”, and “how strong is Vlad’s arm”? (They were tied for 1st). Like I said, you can simply decide to only look at IF-only and OF-only, and say Furcal has the strongest arm for an IF and Vlad for OF, without having an opinion as to which one is actually stronger.
That’s the great thing about this survey, that it doesn’t have to give you one final number. The framework is the framework. How you wish to analyze the results is your own implementation.
Having just finished filling out a ballot for the M’s, I have to say I’m a little unsure how to evaluate the catchers. The throwing categories are pretty straightforward, but the others are problematic when applied to the guy behind the plate, it seems to me. “From pitcher to crack of the bat” is what for a catcher, blocking pitches? And since the only fielding they do is catching pop-ups and the occasional play at the plate…
We mortals could have a good time watching two brilliant baseball minds discuss defense.
TT: If you have an issue with the way Bell is perceived, it’s not with the fans, but with the numbers.
versus
DC: So, pretty much every piece of evidence points to David Bell being a terrific defensive third baseman. You might want to reevaluate your position.
Which has more validity and reliabilty: the fans or the numbers?
SS: it’s kind of a NAFTA joke (pronounce it like a Mexican to make it sound like a Canadian).
joser: I know, the catcher one is a bit tough. I put blocking pitches with “hands”. His first step I’d put on bunts. His sprint speed on chasing down flyballs.
As for instincts, it probably doesn’t apply to them too much, but is a good place to capture “intangibles” or perhaps “framing pitches”.
The catcher to non-catcher comps are the most problematic, and I really ought to do something to fix that.
To clarify what I meant by “stereotype” a little bit:
Let’s take one trait — arm strength.
There is likely a certain “minimum” rating that clubs will tolerate at 3B. Ideally the player would have a “plus” arm (about a 60). Players attempting to play 3B with “40” arms will tend to be weeded out.
This is where I was going with that: The positions in the field require (or strongly encourage) certain skills are in and of themselves self-selecting for certain traits. That’s why I figured all the Catchers would look loosely the same, as would 1B, 3B, etc. Hence the term “stereotype” or just “type”, for lack of a better word.
#17: that’s the cool part. I once ran the 03 scouting data and the 03 UZR against the 04 UZR. The correlation coefficient (r) was something like .35 for the scouting data and .50 for the UZR data. And when I put the two together it was .60.
If they were truly independent, I would have expected sqrt(.35^2+.50^2) = .61 . So, a very impressive result, that fans were able to divorce themselves from the numbers.
As you can see, the numbers have more weight, but the fans still had alot of added value.
As I mentioned in the other thread about an Ichiro .350 being different from a WFB .350, the fans will be able to pick up aging patterns probably better than a standard aging curve.
So, for David Bell, even if his UZR numbers are remaining strong, you have to remember that the numbers are just samples/observations of his true talent. It needs to be regressed. Rather than regressing him to the average 3B, you’d regress him to the average 3B who is in clear physical decline.
Cleanup: (apologies)
This is where I was going with that: The positions in the field require (or strongly encourage) certain skills. [Positions] are in and of themselves self-selecting for certain traits. That’s why I figured all the Catchers would look loosely the same, as would 1B, 3B, etc. Hence the term “stereotype” or just “type”, for lack of a better word.
This is the point of the survey, to actually figure out what it is that fans see, and we can keep that as a(n) historical record.
I’m delighted to see this. It’s a new, useful, and different way of doing history. Thank you, TT.
french: you are right that the self-selecting process would ensure that the variation of 3b arm strength will be low compared to the other traits, and a fan will be predisposed to rank a 3b arm strength at least average. In the link above on 3B, I also showed the standard deviation. The spread in arm strength was half that of the other traits, even though there were other traits that had the same mean. Whether this is caused by the higher difficulty of judging hands than of arm strength, or because of the fans’ “a priori” of expecting a strong arm at 3B is a good discussion topic.
18 – Joser that is too funny! My tongue had a cramp trying to pronounce it that way. Thanks a lot.
#21
You’ve made something complex, very simple. You ought to teach.
Still, and I know I’ve tried to understand regression before so forgive me, what does regression do in the case of David Bell?
Tango’s rating system seems decent, but it’s hard to assess. I usually think of guys relative to their position. Mind attack…
26, thanks.
I recommend this link:
http://www.socialresearchmethods.net/kb/truescor.htm
And in there, there is also a link to “reliability theory”. I recommend you read both pages.
As for David Bell, specifically, I don’t have the numbers.
And this one too, which explains Regression Toward The Mean:
http://www.socialresearchmethods.net/kb/regrmean.htm
tangotiger…
I would like to fill out your survey, but given the small sample of responses you’re likely to get, I worry about my biases skewing your results. Is that just part of the deal, or should the idea that I may over/undervalue an individual or two keep me away?
I guess what I’m asking is how skilled an analyst do you want a respondent to be?
If you’ve seen 10 games, and you love baseball, then you’re in.
I doubt you will skew the results, especially for the Mariners. Last year, I had some 40-odd ballots for Seattle. Even if you give Ichiro a “1” for his arm, and the other 40 give him a 4.5, the average of the 41 ballots is now 4.4.
I went ahead and filled this out.
I hope that “not sure” answers aren’t too much of a problem; I’ve been a bit of a dilettante about watching games this year, and couldn’t (for instance) say anything reliable about Rene Rivera’s defense.
I filled it out too. When it came to Sexson’s arm questions I put “Not Sure.” I’m sure I’ve seen Sexson throw dozens of times in person this year but it just didn’t make an impression on me.
Couple of thoughts:
1. Would you suppose that since he plays 1B I just sort of block his throwing out of what I pay attention to? The reasoning being a 1B’s throwing ability isn’t going to effect the outcome of the game much(?)
2. Could it be that he has a really “average” arm and nothing about has really caught my attention, so I never make a mental note of it?
I’d be interested to see the correlation between the “Not Sure” answers and an average result of “3”. (In other words, I think there’s a possibility that the closer the actual answer is to “3”, the more “Not Sure”‘s you’ll get.
The only time I think the survey’s in real danger from subconsious position adjustments is if a guy is really unusual for his position.
Like Johjima. He’s got decent speed. But for a catcher, he has amazing speed. I think fans might tend to rank him as having good speed, even though he’s probably more like average. But because he’s so much faster than one might expect (given his position), his performance gets overvalued.
But that’s going to be a fairly rare problem (if it even exists). Tango has a great system, and now he’s building a pretty handy historical record.
I wonder how many people are goign to go on there and give Bloomquist all 1’s.
See, Johjima’s an example of how speed and quickness are totally different, I think. He’s pretty quick behind the plate – he moves his feet faster than most catchers, slides from side to side okay (when he wants to and isn’t stabbing at the ball, anyways), and can get out of a crouch without much trouble.
But man, when he’s just running the bases, he’s so freaking slow, it’s amazing. He’s probably the slowest baserunner on the team. Watching him run out close plays at first base is comical.
Johjima, to me, is quick, but not fast at all.
I didn’t rate Joh’s hands very highly, due to all the balls that get past him. I’d imagine the “stabbing at the ball” you’re descibing is responsible for a lot of that. Maybe I should have marked down his Instincts instead.
On the subject of Sexson…
It was interesting, and this says as much about me as anything, I guess… but my opinion of Sexson shot up the day Broussard played first. Not quite that, but it was a good example of how hard it is for me to judge first basemen.
For example, it took my a while to realize that Olerud was not just a good first baseman, but a SUPERB first baseman.
Sexson has been surprisingly reliable at first. And when I watched Ben’s first relay to second land about fifteen feet short, that I was really surprised that Sexson did that… then I realized Sexson wasn’t playing first and that set off this whole rambling chain of thought…
Yeah, and Ritchie always seems slow the way a galloping giraffe seems slow, because those long limbs seem to be moving in slow motion. Then you notice the background going by and realize he’s covering a lot ground. But he’s still pretty slow, compared to say your typical centerfielder. He gets a pretty good first step, though, just because it’s so huge.
But there are balls that get by him (there was one last night) that wouldn’t get past Beltre, or most 3rd basemen. And I’ve seen Pujols snag drives like a 3rd baseman, balls that hardly any other 1st basemen would get to. But Sexson has huge value as an enormous target for the other infielders to throw at; that was key to Olerud’s value too: he made the other players better.
I’m definitely not comparing Sexson to Olerud, just saying that I think I undervalued him somewhat defensively… because while I agree his range and acrobatics aren’t good, he does seem to be very… reliable is the best word, I think. If he needs to catch a ball thrown close to him, or throw a ball to second, I just assume he will, no worries. That’s worth something.
In the years I’ve done this, I think the most ballots I’ve had for a team was around 110 or 120 (Sox, Yanks).
I’ve already got over 160 for Mariners, and it’s been two days. I didn’t think I’d see a site blow SOSH out of the water…
I don’t think Sexson makes the other infielders better. Sure he’s tall, but his hands aren’t that great. He’s no good at picking the ball in the dirt like Olerud was.
Of course Olerud was amazing at it so it’s not really a valid comaprison, but I still don’t think Richie is that good at it.
It was fun to even think about the players in that context, Tango. I’ll be very curious to see the results.
36 – Yeah, Dave, but I watch Jays games. I get to watch Bengie Molina “sprint” at maybe half Johjima’s top speed.
That Bengie Molina exists means I can never rank anyone “Poor” for speed, because they’re all so much faster than Bengie.
I’ve already got over 160 for Mariners, and it’s been two days. I didn’t think I’d see a site blow SOSH out of the water…
Don’t underestimate the steamroller that is USSM.
Don’t underestimate the steamroller that is USSM.
Indeed.
That picture of Doyle? Nearly 2000 views now, or 1/7th of my total Flickr views.
The other 6/7th is my mother looking at pictures of her granddaughter 200 times a day. And 1 or 2 decent shots of non-family.
One of the neat things about this system is it basically uses a distributed network to create its data, so it’s probably surprisingly accurate.
One thing that jumps out at me, though, is the results for superstars and hyped defensive wizards might be less accurate than for regular joes like David Bell or Sexson or whoever. For instance, Jeter’s a guy who seems to be overrated defensively, and while there’s been some backlash against that, it’s possible that most of the people submitting ratings for Jeter (i.e. Yankees fans) may be more likely to buy into the perception that he’s great.
Or like, as an M’s fan, I may be inclined to overrate guys like Ichiro and Betancourt because I’m always hearing about how great they are in the field. It may not be a big distortion, considering they ARE great, but when all I’m hearing is how strong their arms are, how fast they are, etc., it’s harder for me to be objective about just how great they really are than for someone like Lopez or Reed or Ibanez, where my opinions of their abilities are more likely to come from observation than from outside sources.
In light of the purpose of Tango’s work, this isn’t a negative, I just think it’s interesting that our perception of the players we pay the most attention to may be less accurate than the ones we spend less time thinking about.
Sexson has been surprisingly reliable at first. And when I watched Ben’s first relay to second land about fifteen feet short, that I was really surprised that Sexson did that… then I realized Sexson wasn’t playing first and that set off this whole rambling chain of thought…
Amazing. I remember the exact same thoughts. I also wondered what extent of practice Ben was getting in prior to that game, since he really hadn’t been seeing any gametime action on the field at all from what I remember.
I think I gave Willie Bloomquist all “poors” and I rated Beltre kinda high. There’s been a lot of passed balls between Rivera and Joh so that affected my voting.