What's with the downtime?

November 29, 2005 6:17 AM

Matt: There has been lots of downtime lately. What's up?
posted by caddis to Bugs at 6:17 AM (22 comments total)

Not Metafilter.
posted by Plutor at 6:22 AM on November 29, 2005

That sounded bitter. But it wasn't. Metafilter 85% of the day is better than 0%. I am thankful for every moment of not-working I can get.
posted by Plutor at 6:24 AM on November 29, 2005

It was down from the time I went to sleep until the time I got to work. That's the kind of downtime I like.
posted by smackfu at 6:25 AM on November 29, 2005

I should point out that for those of us outside North America, it's kinda been MeFi for 25% of the work-day, no-MeFi for the rest. It sucks.
posted by Marquis at 6:25 AM on November 29, 2005

That's what you get for living on the wrong continent.
posted by Cyrano at 6:26 AM on November 29, 2005

Matt's beta-testing a new feature for Cold Fusion.
posted by mr_crash_davis at 6:27 AM on November 29, 2005

hear, hear Marquis... I did not like it neither.
posted by kudzu at 6:29 AM on November 29, 2005

I just keep reminding myself that at least it seems to be faster than Wikipedia.
posted by selfnoise at 6:31 AM on November 29, 2005

Matt was tweaking some queries and indexes yesterday, which he believed led to more problems, but that would have really only affected the last 24 hours.

Matt, could you make a cronjob to test every 5 minutes to see if the server is down, and automagically restart it? Or is restarting a non-trivial action?
posted by Plutor at 6:39 AM on November 29, 2005

Actually, apart from this morning, it's been very good the last couple of weeks.
posted by cillit bang at 7:08 AM on November 29, 2005

Sheesh - I thought once Fiona was born, Mathowie would never be getting any sleep so he would always be awake to reset the server. This Fiona thing isn't working out the way I planned. *back to the drawing board*
posted by madamjujujive at 7:09 AM on November 29, 2005

I'm going to switch those deep queries back that I messed with yesterday and see if that improves things. Uptime has been pretty good for the past couple weeks. Last night I went to sleep early and it went down soon after.
posted by mathowie (staff) at 8:03 AM on November 29, 2005

wah
posted by Stynxno at 8:17 AM on November 29, 2005

cillit bang - how do you generate them purdy graphs? :) I want some!
posted by dabitch at 8:42 AM on November 29, 2005

"these queries sure are cold."
"yep. deep, too."
posted by quonsar at 9:06 AM on November 29, 2005

developing in production is fun!! ;)
posted by scarabic at 9:35 AM on November 29, 2005

*runs around like a monkey on crack while yelling "developers" and sweating profusely*
posted by keswick at 10:13 AM on November 29, 2005

That graph is rendered by a simple PHP script that uses the image tools and a MySQL database, with a cron job to collect the data. I knocked it all up in about half an hour while MeFi was down. I'll mail you the source code if you're interested.

(It's hopefully server friendly, too)
posted by cillit bang at 10:32 AM on November 29, 2005

developing in production is fun!!

It would be hard to simulate the kind of resource gobbling that's causing the problems in any other environment. Siege maybe?
posted by If I Had An Anus at 10:39 AM on November 29, 2005

Yeah, load testing is hard. I'm just joshing. I happen to work at a large, mainstream web company where uptime is money, and nothing goes out the door without 3-stage testing and full instant rollback in place. I actually find the seat-of-the-pants thing around here refreshing and at times hilarious. But in truth I don't know what kind of testing mathowie does, or what his release process is, so I am not trying to be really critical.
posted by scarabic at 11:00 AM on November 29, 2005

mathowie writes "I'm going to switch those deep queries back that I messed with yesterday and see if that improves things. "

Post SQL. Or email me. And, of course, check your error logs.
posted by orthogonality at 12:48 PM on November 29, 2005

What if MeFi only exists when mathowie is awake? What if I only exist when mat is awake?
posted by plinth at 8:20 PM on November 29, 2005