Yup, ignore ESPN
The Vegas Watch blog evaluated the preseason predictions of a lot of famous people, plus PECOTA’s projection system. And then Tom Tango updated the chart to include the default projection of every team going 81-81. Here’s the revised list (number to the right is RMSE)
9.6 PECOTA
10.2 Neyer
10.5 Law
10.6 Perfectly Competitive Balanced (all teams predicted at 81-81)
10.8 Vegas
11.1 Passan
11.3 Sheehan
11.4 Brown
11.7 Kurkijan
12.1 Stark
12.1 Henson
12.4 Phillips
13.0 Olney
PECOTA did the best, and then two PECOTA-ish writers (Neyer and Law) also beat the default projection. Everyone else did worse.
Let me repeat – everyone else did worse than predicting all 30 teams to go 81-81. If you knew absolutely nothing about baseball, didn’t know the rules, had never heard of any players, but you had a basic understanding of probability, you would have done better at predicting the 2008 MLB season than pretty much all of the experts on TV.
Yep.
Keep this in mind the next time some professional baseball writer calls you an arrogant bastard for daring to ignore their insights and opinions that stem from years of experience. They suck at turning their experience into actual knowledge. In fact, they suck at it so badly that they can actively take away knowledge. Listening to Steve Phillips and giving him credit for knowing something can actually make you know less about baseball than if you stuck your fingers in your ears and hummed every time he came on TV. And he used to run a baseball team.
People who think that their positions entitle them to have their opinions respected are likely not worth listening to. If they actually had something of value to share, they wouldn’t have to lean on their employment status, but could stand on their track records instead.
Moral of the story – ignore the media. They don’t know baseball any more than you do.
Great link. I loved the Vegas Watch comments on the Seattle prediction by Steve Phillips, which they ranked as the worst of the year (he picked Seattle for 92 wins).
This reminded me a lot of the success rate of managed mutual funds in beating the market over the long term (they don’t)
Although they gave the outliers on a number of club’s projections, I’d also like to see how much they missed in projecting the Yankees this year. BP was very optimistic that Hughes and Kennedy would be great this year, but what saved the Yankees from disaster in their starting pitching were the three old guys, Mussina, Rivera and Pettitte.
So does one year make a trend?
Interesting insight, but I don’t think it’s complete until you have 5 to 10 years of this tallied.
What if Phillips’ predictions were number 1 in 2007? Then what conclusion do we come to?
I remember Philips raving about the M’s defense in spring training. Thankfully I didn’t have any hammers with me.
I enjoy hearing the media guys talk about anecdotes and the inside stories they have access to. But for hard-boiled analysis…
This isn’t new – the talking heads suck at this every year.
As it happens, Phillips had the worst ranking last year, behind Olney (though both did worse this year than either did last year).
Jesus, look at Olney.
Would love to see what Gammons did, as well. Maybe he didn’t have predictions available. Was the start of this season when he was laid up in the hospital?
Hey, that’s why they play the games….
Shrug.
Reporters are very good at the descriptive. They’re not very good at the predictive or the analytic.
Worth reminding – Mike Greenberg’s (of ESPN’s “Mike and Mike in the Morning”) 2008 preseason World Series Champions: your Seattle Mariners. Then Jayson Stark at least agreed that it was plausible.
So how accurate was PECOTA this year? This result only tells me that PECOTA was better than sucky competition. A random coin would beat the talking heads. I don’t think it is safe to take PECOTA to Vegas and start making bets.
It wouls be ineresting to run the numbers on predicting the previous year’s win total.
By definition RMSE would punish a prediction system that had a large variance. A prediction system could do very well on many picks and then miss a couple by a large amount and lose to the 81-81 system. Perhaps there should be alternative measures used to determine which predictions faired the best? 81-81 beating Vegas raises a red flag for me.
vr, Xei
The problem is that Vegas lines reflect fan sentiment. Fans generally bet on their teams for pennants/titles, and bet the over on win totals. I remember working out the overall odds before a season and finding the average team was predicted a couple games over .500.
Which in turn reminds me of something Neyer did once, where he showed that analysts making team-by-team predictions tended to show the same thing, because they’re not good at seeing where teams might get worse.
ESPN: It’s like People magazine for sports fans!
ESPN used to put up a page with their picks, and we used to go back and check on their futility at the end of the season– and it was futility …
ESPN has a page for Football picks and I’m pretty sure I saw picks change after the game was over to make the analysts records look better.
If one goes back and looks at the closing prices for team win totals, one will note the most expensive “over” was for Tampa Bay, and the most expensive “under” was for Seattle. In both cases, the marketplace corrected for a unrealistic expectation for each team. If comparing PECOTA or ayone else to team win totals, one must also factor in the price.
Did any of the authors of this blog make pre-season win total predictions? It’s easy to make fun of other people’s poor predictions, especially when you didn’t make any your self. Not everybody makes nor cares about making predictions, but being able to accurately project how many wins a team will get based on such methods as WAR are important and were a steady topic here during the pre-season and Adam Jones trade talk days. Let he who beat the 81-81 line be the first to cast a stone.
vr, Xei
why, yes, yes they did.
sigh. remember Running the 2008 season a hundred times?
Your 2008 simulated Mariners:
Average record: 77-85
Average runs scored: 716
Average runs allowed: 759
Number of times they won the division: 6
Number of times they won the wild card: 0
Best season: 93-69
Worst season: 59-103
Best offense: 804 runs
Worst offense: 607 runs
Best run prevention: 674 runs allowed
Worst run prevention: 866 runs allowed
Heh. Heh heh. Heh heh heh.
(addendum: I got the distinct impression the numbers were pushed UP a tad to make them seem more acceptable…but I could be wrong….)
“Did any of the authors of this blog make pre-season win total predictions? It’s easy to make fun of other people’s poor predictions, especially when you didn’t make any your self. ”
Did you bother looking at the archives before you asked? Doesn’t sound like it from your tone.
When you assume you make an ass of … well, just yourself really.
(addendum: I got the distinct impression the numbers were pushed UP a tad to make them seem more acceptable…but I could be wrong….)
Well, if you go back, you see I bumped Betancourt’s defensive rating up, which in retrospect is embarrassing. It didn’t end up being that big a deal.
And there was another set of projections later where I went through and crippled the A’s pitching staff.
yeah.
that was fun.
I am not sure if the authors consider that DMB run their predictions or not. If so, they might not have done as well as the 81-81 system if the A’s came out on top most of the time. The Mariners win totals were probably closer than most of the pundits. Really, my only point up above was that if you are going to slam/make fun of somebody elses picks/predictions, why not include your own for public inspection? Especially, since the 81-81 method was made up after the fact. After knowing that an RMSE was used. vr, Xei
It wasn’t. I predicted 83-79, though you can go read my hand-wringing on the why. I pretty much always find a way to predict an 80-win season. You can search for this stuff. We’ve got five years on the record here.
Also, I don’t see how our one-season predictions enter into an all-baseball prediction set. Or why you seem to think that we made up a method.. actually, I’m not sure what you’re saying there. Dave’s quoting someone else’s work, if that’s not clear.
DMZ, no biggie. I know he is quoting Vegas Watch’s (a great site) work. I just didn’t think it was fair for him to lambast the pundits who were outperformed by the 81-81 method the way he did. He’s preaching to the choir on the overall message, I just didn’t think it was fair to put them down the way he did without adding his own predictions into the mess. Yes, many in the media are “stupid” when it comes to baseball analysis, but without beating those he calls “stupid”…???
vr, Xei
Did any of the authors of this blog make pre-season win total predictions? It’s easy to make fun of other people’s poor predictions, especially when you didn’t make any your self.
And it’s easy to make an ass of “your self” by casting stones without lifting a finger to do a little searching in the archives.
And it’s easy to make an ass of “your self†by casting stones without lifting a finger to do a little searching in the archives.Quoting Shakespeare again?
vr, Xei
Of course. Joser is a learned and erudite man.
A word to the wise, and all….
I’m guessing the 81-81 prediction doesn’t fair quite as well in most seasons. As VegasWatch pointed out, this was a very difficult year for predictions, making a .500 record prediction look better.
Two other mindless predictions to use as baselines would be last year’s record and last year’s Pythag record.
If anyone entered the BPro Predictatron contest, you can look up your own pre-season predictions and compare to the list.