Community Projections
In 2004, James Surowiecki published a book entitled “The Wisdom of Crowds”, in which he discussed how a well rounded group of laymen were better at evaluating almost anything than any individual expert in that field. Essentially, he posited that a large group of people would almost always do better in any test of knowledge than an individual person, even if that person was clearly more informed than any other individual in the crowd.
This applies to almost every facet of life, including baseball. As notable analyst and friend of USSM Tom Tango has shown, an amalgamation of projections from fans often perform as well or better than advanced projection systems or baseball experts. By combing the various bits of knowledge from a large sample of people, the inherent bias’ that each of us hold can generally be overcome in most cases, presenting an accurate assessment of reality.
In that spirit, we’re happy to join forces with Jeff Sullivan and the gang over at Lookout Landing to present the 2007 Community Projection series. Last year, Jeff spearheaded this project on his blog, and this year, we’re joining forces to give the Mariner blogosphere readership a real chance to show your stuff. Is the group widsom of the average USSM/LL reader more knowledgable about the potential of this team than advanced systems like PECOTA? I say yes, and this is a chance to find out.
The community projections officially kick off on Monday. In order to make the collection and calculations easy and accurate, we’re going to be utilizing Google Spreadsheets. This will allow us to share an Excel style workbook over the web for collaboration and publishing. To use Google Spreadsheets, you need to have a (free) Google account and join the community projection mailing list so that we can add you to the list of collaborators.
We will not send any spam, sell your address, or do any other annoying things – we just need to be able to invite you to edit the spreadsheet. To participate, send an email to communityprojections@gmail.com. On Monday, we’ll send a notice inviting everyone on the list to enter your projection for Kenji Johjima, and we’ll proceed around the diamond with the rest of the team over the next several months.
You don’t need to be an expert to feel like you have something to offer. If you’re reading the blog, you’re probably a pretty well informed fan, and we’d love to have you participate in the community projection series. So, sign up today and help us make this a true test of the wisdom of the Mariner blogosphere crowd.
I can’t wait until the projections for Wille B. His numbers should be off da hook!
Can someone print or is there a link to show all of last year’s predictions and the final stat lines?
In 2007, I project Willie Ballgame to stop the genocide in Darfur, produce the next three “Star Wars” movies, and invent an automobile that uses garlic fries for fuel. The true value of some players is impossible to express in statistics.
Nothing better than Baseball Stats and Spreadsheets! WOO!
The book was really great. For anyone who hasn’t read it they should pick it up. Has (to me) a really interesting way to clear up rush hour traffic in Seattle…
Wait, if garlic fries are used for fuel wont that raise their cost!?
#2,
in the archives at Lookout Landing Jeff has a write up on each position and how the projections compared to actual
I did something like this last year, using 6 different sources of projections (Bill James Handbook, PECOTA, Ron Shandler’s Forecaster, ZiPS, Fantasy Sports Central, and Rototimes). The average of all 6 projections did in fact finish with a higher correlation than any individual source of projections in the categories of OBP, SLG, ERA, and WHIP.
The “community is smarter than the individual” idea is a great theory. But doesn’t it seem that there is an inherent bias in this case? The community of USSM/LL readers probably doesn’t represent a “well-rounded group” regarding the performance of the Mariners. It seems to me that the vast majority of this group are fans of the Mariners, and want them to win. It seems that would inherently affect their projections.
I guess the main hope is that the part of the community who’s fan-dom causes them to overproject is balanced by an equally large part that causes them to under-project. In any case, this hardly seems like a neutral application of the method.
#9: After the off-season we’ve had, I think you may find the bias to be on the pessimistic side! Especially when it comes to Vidro, Ramirez, etc.
How did last year’s community projections do in comparison to others?
Okay…I’ll admit it. I just skipped write to the bottom after reading and posted my comment…then went back up to read the other comments and saw my question (and the answer) was already covered. Thanks everyone!
I agree with Hans. Whenever I’ve seen anyone throw out projections for Mariners players, they have typically been on the high end.
Also, I think that most people (myself included) who make their own projections look at the advanced systems, decide whether those are high are low, and make adjustments from there. If it were possible, I’d like to compare the aggregate results before and after the projections come out from Baseball HQ, BP, BTF, etc. Not sure how we’d ensure that we had a representative sample in both of those sets, though, or that other news and noise hadn’t caused any differences that might be manifest. In theory, though, I’d hope that the earlier set might give us some insight as to what fans expected based upon their own methods (or lack thereof), completely separate from any of the advanced systems.
Yes, there are flaws with our “crowd”. Most of us are all influenced by common things, such as writings on USSM/LL and information from other projection systems.
Even with these flaws, I’d bet our crowd is smarter than PECOTA by itself. That doesn’t make PECOTA useless – it is one of our inputs, after all – but it also makes the effort worthwhile.
It’s not a perfect experiment, but it’s a useful one nonetheless.
#9 and #13: Actually, the author of the book would state that it really doesn’t matter. As long as we are diverse enough in our own opinions and there are enough of us it should all cancel each other out. The only argument against the crowd would be if we felt like we all had to agree with certain points (kind of like Bay of Pigs) but I doubt that would happen
What would be an even more interesting experiment would be to create a “market” where fans could place buy or sell orders based on certain projections of players. That would probably give you even more accurate projections…and would probably be pretty fun.
#9 and #13…Also, the author also strongly supports the idea of corporations using their own employees to help make key decisions (especially by creating a decisions market). One would argue that using people all from the same corporation would be biased…but it has proved that it is not.
So, we should be pretty good.
It’s not a perfect experiment, but it’s a useful one nonetheless.
I think that viewing it as an experiment is a good attitude to take here. Even if it turns out that PECOTA does better than the wisdom of crowds, that would be an interesting outcome.
#13.
I agree, although I wouldn’t say “anyone”, I’d say “Mariners’ fans and denizens of Mariner blogs”. The non-Mariner writers and sites seem to be more pessimistic in general, maybe even too pessimistic regarding the Ms. (I’m thinking mainly of the seasons from around 2000 to 2003, when pundits thought that Ms were getting too old to continue to succeed. Eventually they did get too old, but it took longer than some people thought.)
Or even see the results of the first round of community projections, and let people adjust their projections and submit new ones, after having seen other people’s projections. This is called the “Delphi technique” in forecasting. College football and basketball pools are somewhat similar, in that voters can adjust their votes week after week — in that case however, there’s also a flow of new hard information, namely the results of the latest games.
My knee-jerk reaction is that the community projections are likely to be of higher quality if people have stronger incentives to submit good forecasts, either in the form of a prize for best projections, or by having to … well, bet in a way; to put up some of their own money. The Iowa Electronic Markets work on this principle, people buy “futures” on e.g. what they think the outcome of the next presidential race will be. These markets have shown good quality results, a well-run stock market or futures market is probably the ultimate example of the Wisdom of Crowds.
Actually, market experiments have shown that greed can overcome wisdom, and that people will continue to buy when they see the price increasing even when they know the price is unsustainable. This is how bubbles (and crashes) happen. Crowds are not wise when greed overcomes fear.
That said, I wouldn’t worry about this particular experiment being too biased or inaccurate. For every booster root-root-rooting for the home team with inflated estimaes, you’ll have someone like me who thinks nobody but Ichiro will break the Mendoza line and projects the team to win 50 games, tops. I’ve reached the tipping point where I’m going to wish for, and enjoy, the mounting losses more than the victories. I feel exactly like I did with the Mariners in the early 80s, just before I quit following them for almost a decade.