Day 2 back up
The first day up facing traffic and we did fairly well. The game thread got up to 150 and I saw some delays, but it never took more than a couple of seconds to get back to me, and we didn’t see any timeouts, which is good.
That said, I have to admit that I was nervous all day as the delays went from 1s to 2s to 6s. And for a 150-comment game thread? I’m not convinced we’re safe yet.
In any event, I’m going to be doing some tinkering with the site to try and help, and we’ll see how tomorrow goes. If you notice anything wonky with the site behavior in the next couple of hours, please drop me a line and let me know what happened, with as much information as you care to include. Or throw it in the contents there.
Like, for instance, seeing a blank page the first time you load anything and only getting served a real page on reload. That would be a problem I’m having right now. WTF is that about?
Update: seriously, wtf?
Update: That’s it, I can’t easily figure out why caching results in blank pages being served the first time, so I’m turning it off while I research
In the meantime — the plan’s to take the donation buttons down tomorrow and see what we want to do from there, so if you want to support USSM with cash, here’s your chance:
Read more
What happened, how to help
Wednesday we got overwhelmed by a traffic spike at least 50% off our normal traffic, which is already shockingly large. We blew up our web and database servers (and that’s “the massive shared servers our generous host had us on”) For a couple of hours, we were showing a version of USSM that was the last thing the web server managed to generate: a text-only version of USSM with no formatting, where every link led to a file not found error (because they’d been blown away).
This meant that while I was on the phone with our host, discussing what in the world we could do (because bringing the site back up only to get destroyed under the load again made no sense) we got high-quality emails like
What Happened to the USS Mariner web site today? New format is awful. . .and nothing about trade that just came down with Indians
No really, someone looked at that broken, formless thing and thought we were redecorating.
Helping
Probably… 95% of the email that’s come in has been requests to help in different forms, and particularly, requests to accept donations. Some of them were rather angry about not having that option. We’ve resisted this in the past, and now we relent:
And also this.
Wow, that Paypal box is way smaller. Oh well, there’s not a lot of time right now for tweaking and formatting.
Paypal (now a part of eBay) is, admittedly, uh, not the finest company, but they do charge us a little less compared to Amazon (PayPal is, I believe, 30c+2.2% while Amazon is 30c+ 2.9%) PayPal’s a little bit better, but Amazon is likely to be much easier for you if it was up, which it hasn’t had much luck at today. If any of our fine readers at Amazon want to kick their box in the hind end for us, that’d be helpful.
I fully admit that we don’t know what we’re going to do with the money right now, but the top candidates are:
- We port the site to use Movable Type, which I believe may be far better at handling the kind of loads we see. MT requires money
- Putting a box on a rack which is not hand-assembled by me with a Will Clark sticker on the case (for luck)
- Bribe our host
I would say right now that I’m 100% certain that we won’t be using the DMZ-built box as a long-term solution: either we’ll move to MT and solve the load issue that way, or we’ll be forced to buy or hire our own server. I promise not to buy my weight in Anchor Steam and try to forget all this when we’re up and stable. Unless that’s a request. Because I could do that.
Some people have suggested Google’s ad program — we’ve been down that path before, and it didn’t do it.
Black USSM shirts! Finally! Woo-hoo!
Took long enough, but they’re here. Those fine JoelE logos sure look good on black. I really wanted navy blue, too, but Cafepress hasn’t responded to my carping yet.
They’re $24, which is crazy… Cafepress for whatever reason set the black base price at like $19, and the standard USSM markup ($5 to the beer and pizza feed hosting fund) puts them significantly over $20, which kinda sucks.
Anyway, check them out.
We’re up
Enough said.
Minor tweak
If you happen to have witnessed the ongoing battle with the polish spammer(s) the last couple of days, it looks like I have prevailed. If you see anything wonky about the comments/registration, drop us a line. And if you didn’t see any of it, that’s good.
Friday cat blogging
Nearly the last picture my Minolta DiMage 7 took before it died. It was a good camera in the rare times it was working.
Drinking guide re-write looms
Hey all — this came up at the last Feed (and comes up fairly frequently in email and people stopping me randomly) so I wanted to say — I’m going to try and get through a total, top-to-bottom rewrite of “Drinking at Safeco” over the course of the next homestand.
What I need from you, dear readers: if there’s a place within walking distance of Safeco you want included, especially if it’s got good drink specials, drop me a line or leave a comment here with name of the place/address/why it’s good.
Outage
In the future, I promise to refrain from praising my host and thus attracting the attention of whatever the digital equivalents of injury spirits are.
Still, better the site than Snelling.
On the new digs and USSM in general
The plan was to scrape enough money out of T-shirt sales, Google ads, Amazon affiliate links, a complicated Moose-napping scheme and whatnot to afford to move somewhere swanky that could host USSM without falling down (this costs more than you’d think) while still having a cushion of a couple months hosting bills. At the rate we were accumulating money, it was going to take us another couple months to get there (T-shirt sales went well initially, but the Google/Amazon stuff was barely going to pay to keep the doors open, no matter my ad tinkering).
So enter digital.forest.
They’re local. And for no reason I understand, they’re helping. After much discussion of what we needed and our current problem, they offered us a deal we could not turn down, no matter our meager bankroll. So we moved ahead of plan, which is fine with me, because that plan meant we were playing chicken with the user exprience: while picking up users and growing quickly, the site slowness and server errors would make it harder and harder to come and read. I was worried, to paraphrase Yogi, that we’d soon be so popular that no one would come here any more because it’s too crowded.
We don’t have a Beowulf cluster of supercomputers here at USSM Labs, crunching stats and inventing new defensive measures on its own. But this so far seems much, much better than our old digs. I don’t think the site went down during the game thread tonight (though, in fairness, not everyone’s over here yet due to the way server change information gets spread). That makes me grin stupidly. You have no idea what a pain that was.
We also have a lot more bandwith to play with, which hopefully means we can do some more photos, and we’re still hoping to get some podcasts up so you can listen to us joke around while riding the bus to the Mariners game, or… I don’t really know. We’ll think of something.
We’re still not corporately sponsored, we still aren’t part of an affiliate network that subsidizes our bandwith and hosting, we still don’t charge for subscriptions or premium content. We’re Mariner fans that have been at this now for three years (and change), and that’s all. I have no idea how much time I’ve spent on the site between writing, fending off comment spammers (and censoring dissent!), and trying to keep the old thing from falling over by doing ridiculously stupid tricks. Including research time on stuff like the Attrition War articles, it’s easily a year’s full-time work since we first threw up a post. I’m not alone — Dave and Jason have been here since the start, and Jeff had another Mariner labor of love before this.
Sometimes, it feels like one of the best things I’ve done, and sometimes, if I’m honest, it feels as if it’s been entirely unrewarding. Mostly it feels pretty cool. We’ve been paid back in random generosity through responses to technical issues, or someone picking up on one of us needing a job and dropping us a hint, or someone offering us a spare ticket if we want to see the game, as if we’re a good friend. Readers want to sit and buy us beer before games, and we have events where a hundred people will show up to talk to other fans and the expression on their faces talking to other smart Mariner fans is like they’ve had their first drink from the oasis after crawling around the desert for days with vultures circling. It warms my cold, antisocial heart.
I don’t know what happens from here, or how long something the current state can hold. We’re trying to do something strange that requires a balancing act I didn’t really anticipate: maintaining the level of discussion requires more intervention the larger we get, and now we have registration. But when the PI shut their forums for a while, I was shooting comments as they popped up, like a carnival game. I don’t know that discussions scale.
I worry that someone’s going to give Dave a full-time job to write about baseball or go hunt prospects, and Jeff’s going to be given some huge book deal to roam the earth and write about his crazy adventures (not a novel — they’ll be well-told stories), Jason’s going to open a restaurant, or (and) I’m going to wake up one morning and find the comments overrun and decide to go work on my next book instead of fix it. Losing one of the authors that way would be joyous and a great loss at once. We’ve seen that when we have more time to dedicate to writing articles, the site shines, but that time comes from somewhere, and we do only have so many days in our lives (I have, for instance, ~16,500 days left, and I’m unlikely to die and think “I wish I’d spent more time moderating comments”).
Based on what we were eking out of Google/etc, we’d need to grow another 10-20x for it to be worth it for one of us to quit their day jobs and write full-time (or every regular reader could give us a couple bucks, but I don’t believe there’s a team-oriented site that’s seen a high enough donation % to encourage me to think this is possible at current reader levels — and let’s not discuss that, please, it’s not the point).
I’m also leary of what happens when years of sacrifice by everyone result in one person making money while the others continue to toil, because all kinds of bad things happen. I don’t want to talk about that any more.
For now, though, here’s the short version:
– we have a new host, and by all measures it appears to be awesome
– digital.forest rocks and you should totally go there for your hosting needs
– I’m clearly willing to push their hosting services even though that’s not part of the deal
– we’re going to do some new stuff, like podcasting
– on a purely cash basis, the site’s now about break-even over its history
– the long-term future of the site is, as it has always been, somewhat uncertain
– fewer people visit USSM on a given day than turn out to see the Mariners
– that gap is not that large, and that’s both great and sad
In conclusion, about 1% of our traffic from search engines last month came from people looking for more information on Jennifer Pankratz, Scott Spiezio’s girlfriend (of which he has a tattoo). Thanks to all our readers, regardless of their taste level, for the last three years.
And we’re back up
These are the new digs – I moved servers late last night after taking comments down. I was initially dismayed to find out that despite my “comments down, be cool” post a bunch of you decided that being cool meant a lot of random commenting on that post. Then I realized that I could discard the changes and you’d get what you deserved. In the future I’m going to try to skip that initial moment of hesitation.
Comments are re-opened on the other threads. If you note anything odd happening, email us.
Filing a good bug report, for those of you who don’t work in software development:
– Here’s what I did, so you can reproduce the problem
– This is what happened
– This is what I thought would happen
Fixed:
– the 404/permalink issue.
– Future Forty’s missing, tonight
Known remaining issues and estimated fix time:
Known issues that are not our fault
– RSS or links resolving to the old site