The Great Rewrite?

August 15, 2007 11:00 AM

Given the regular and predictable performance issues with the site, and my limited understanding of the infrastructure behind the current implementation, when is the Great Rewrite planned?
posted by dblslash to feature requests at 11:00 AM (87 comments total)

For the moment we're doing a few things.

- upgrading to the latest ColdFusion 8 which uses a new java architecture instead of the crusty old one with low memory limits that the current release is hindered by.

- we're redoing small bits here and there in PHP. You'll see new features in the comings that aren't stuck in ColdFusion

- A total rewrite is a big job and something we were thinking about doing in a six month or so timeframe if we couldn't get any more big improvements via other means.

So for now, we're still pushing ColdFusion and hoping to make some big gains in the next couple weeks and if the situation is still bad a few months from now, we'll be diving back into a rewrite. It mostly works for now and is better than bad times have been in the past, but is still sub-optimal.
posted by mathowie at 11:12 AM on August 15, 2007

To counter the absurd proposal by the original poster (Complete rewrite?! please...), what's the status on the current performance tuning effort? What are the known bottlenecks? I hear that this crazy thing runs on CF (what version?) against a (reportedly "beefy"?) SQL Server backend (version?). Got any disk i/o numbers on the db? Done any profiling on the app server?
posted by dblslash at 11:14 AM on August 15, 2007

General Haughey is going to report back in a few months, and until then I think we should give the surge time to work.
posted by pb at 11:14 AM on August 15, 2007 [14 favorites]

General Haughey? I thought his name was Petraeus.
posted by Cranberry at 11:17 AM on August 15, 2007

Nonsense. What sort of name is "Petraeus Haughey"?
posted by cortex at 11:25 AM on August 15, 2007 [2 favorites]

The next six months in ColdFusion—which will determine the prospects for performance there—are the most important six months in MetaFilter uptime in a long, long time.
posted by grouse at 11:27 AM on August 15, 2007

Bah! You kids have it easy! Why in my day, we had to walk eighteen miles in the snow to get the Best of the Web, battling JRun errors the whole time, uphill both ways. And there weren't no AJAX, neither, if'n ya wanted a web app you had to refresh the page for each little tiny bit of data. By hand. With a cast iron sledgehammer and a wooden bucket to hold the bits in. And we liked it, by gum! Built character, it did.
posted by arto at 11:37 AM on August 15, 2007 [2 favorites]

Nonsense. What sort of name is "Petraeus Haughey"?

A perfectly good one, except difficult to pronounce. Actually, that's not true, it's easy to pronounce, it's just that no one can do it correctly, given the mish-mash of consonants and vowels handed to them.

He tried "petr_howie" or "petrahowie" and instead had to smash it down to "pethowie." It seemed self-explanatory, but everyone said "Peth Owie? Is that like some sort of injury to an Indian state?" But no, it's just "Petraeus Haughey," spelled phonetically, and crammed into eight letters.

(I, for one, am glad he just gave up and went with his middle name)

* disclaimer: have no actual knowledge of Matt's middle name.
posted by pineapple at 11:39 AM on August 15, 2007 [1 favorite]

But of that day and that hour knoweth no man, no, not the angels which are in heaven, neither the Son, but the Father.

Or something largely to that effect.
posted by dismas at 11:42 AM on August 15, 2007

Seems reliable enough to me. It's just Metafilter, not anything important.
posted by smackfu at 11:45 AM on August 15, 2007 [2 favorites]

Nonsense. What sort of name is "Petraeus Haughey"?


Think of it as having a Patronus Haughey when the JRun tries to suck all the happiness out of the Internet.
posted by jmd82 at 11:46 AM on August 15, 2007 [1 favorite]

I hear the JRun insurgency is in its last throes.
posted by donovan at 11:50 AM on August 15, 2007

mathowie: "we're redoing small bits here and there in PHP. You'll see new features in the comings that aren't stuck in ColdFusion"

Holy shit. This is possibly the only time in history I've been happy to see a website migrate to PHP. Not to belittle Matt, his coding prowess, or ColdFusion, but... no, wait, the latter is exactly what I'm trying to belittle.
posted by Plutor at 11:50 AM on August 15, 2007

Yay, incremental rewrites!

Great Rewrites fail so often that I think the only reason anyone ever tries them is that they seem more fun than trying to fix the problems individually.
posted by aubilenon at 11:54 AM on August 15, 2007 [4 favorites]

peanut butter will fix a myriad of ailments
posted by Cranberry at 12:05 PM on August 15, 2007

As long as the rewrite includes a more professional-looking white background, I'm on board.
posted by Kwine at 12:14 PM on August 15, 2007 [3 favorites]

We've decided that, in lieu of a Great Rewrite, we'll just make the [more inside] links on the front page of AskMe work as links to the thread proper.

[NOT JOKIST], even. Any bug reports are welcome here.
posted by cortex at 12:18 PM on August 15, 2007

Wait, you mean they didn't always do that?
posted by IronLizard at 12:21 PM on August 15, 2007

"Ladies and gentlemen, uh, we've just lost the picture, but what we've seen speaks for itself. Metafilter has apparently been taken over -- 'conquered' if you will -- by a master race of incemental upgrades. It's difficult to tell from this vantage point whether they will consume the server or merely improve upon it. One thing is for certain: there is no stopping them; the PHP code will soon be here. And I for one welcome our new Web 2.0 overlords. I'd like to remind them that as a trusted web personality, I can be helpful in rounding up others to toil in their underground pancake caves."
posted by blue_beetle at 12:23 PM on August 15, 2007

Hell, just reinstate the draft!
posted by R. Mutt at 12:26 PM on August 15, 2007

How much of the load is retrieving data for presentation/refreshes, and how much is actual writing (i.e., comments/posts and user stats such as last time visited, etc)? If most of the activity- in bandwidth, and in CPU- is read-only retrieval, couldn't we really extend the power of Metafilter by pushing that load out to a network of P2P web hosts?

Perhaps in lieu of a rewrite, a distributed architecture of volunteer proxying servers which cache via something akin to feed reading the actual data of the system. I.e., if instead of re-writing the system, a semi-secure link to get the newest data in say XML format could be presented to participating hosts, who could then be who the end clients talk to? For writing/posting, they'd go back to the "real" server, but for reading/navigating the proxying servers could handle this load? You could also use that to beef up the search capabilities as well...
posted by hincandenza at 12:27 PM on August 15, 2007

I thought "The Great Rewrite" was something that went along with believing in Intelligent Design. Are you some kind of Creationist, dblslash?
posted by wendell at 12:28 PM on August 15, 2007

If a re-write is necessary, there will be JRun rationing in the meantime: those with an odd number of vowels in their usernames will not be able to access the site during peak times Mondays and Wednesdays. Those with an even number of vowels in their username sare out of luck Tuesdays and Fridays. Those with no vowels in their usernames lose out on Thursdays.
posted by mikepop at 12:35 PM on August 15, 2007

Suddenly the front page shows an argument AGAINST a Rewrite: Efficiency Maximizes Catastrophe!!!
posted by wendell at 12:39 PM on August 15, 2007

When is your mom's Great Rewrite?
posted by SassHat at 12:49 PM on August 15, 2007

...their underground pancake caves.

Sweet! Sign me up!
posted by philomathoholic at 12:56 PM on August 15, 2007

hincandenza, overkill alert. An ad-hoc p2p network will respond quicker than a single server? Writing the system to manage XML strewn across various CDN-style mirrors will be easier than just slapping together the php? Most performance bottlenecks on massive sites are solved by doing less DB hits (memcache, disk cache etc.) not by somehow spreading frequently-updated data around different hosts!
posted by Firas at 12:59 PM on August 15, 2007

Can't we just, like, y'know, upgrade from the PII 200Mhz system mathowie's got going in the basement to like, a totally sweet rig with blue neon lights?

That would solve a lot of problems, right?
posted by djgh at 1:16 PM on August 15, 2007

I'm pretty sure this site would be faster if we just implemented some Bit Torrent. That way I could just use one interface for both my MeFi and my porn.

Wait. That might not be a good idea. I don't want to mix up my porn and cortex.
posted by eyeballkid at 1:19 PM on August 15, 2007 [1 favorite]

* disclaimer: have no actual knowledge of Matt's middle name.

I have it on good authority that it's Englebert-Slapdyback.

'Good authority' here means the voices in my head.
posted by quin at 1:23 PM on August 15, 2007

I don't understand. How do you rewrite Usenet?
posted by srboisvert at 1:24 PM on August 15, 2007

Wait. That might not be a good idea. I don't want to mix up my porn and cortex.

You've got your porn in my cortex! You've got your cortex in my porn! Mmmm, it's Cortex's pornybutter cups!


Firas: actually, yes, it will be more performant. Pish tosh! This is why this company can and does charge boatloads of cash for their services: the tcp connections, the data offloading of frequently requested information, etc all can free up the core dynamic content servers to be optimized to do work better, and to scale much faster and farther to load and increased demand.

Imagine how well Metafilter only ever had 10-20 people browsing it at once- that's what reverse proxying and caching systems can effectively emulate. The point is that the data is frequently appended- meaning new posts, new comments- but not frequently updated for the most part.

I'm not saying it would be trivial work, but if designed and done well, all Petraeus would have to do is implement a couple of new APIs, and the offloading could be done separately. It would be fascinating- a sort of Coral Cache for very dynamic, real-time, frequently updated websites. Has that ever been done before?
posted by hincandenza at 1:27 PM on August 15, 2007

I don't want to mix up my porn and cortex.

I dunno, I've been thinking about getting into soundtrack work...
posted by cortex at 1:30 PM on August 15, 2007

When is your mom's Great Rewrite?

Yeah, I rewrote your mom last night.
posted by Slarty Bartfast at 1:31 PM on August 15, 2007 [1 favorite]

- we're redoing small bits here and there in PHP

The performance gain may be good, but PHP is the biggest security hole on a server so please consider using some variation of MAC on the backend and using stored procedures to prevent SQL injections, to save yourself a nightmare down the road.
posted by cmonkey at 1:31 PM on August 15, 2007

all Petraeus would have to do is implement a couple of new APIs, and the offloading could be done separately

I'm afraid that, in this operation, the apC has proven more a liability than an asset due to it's lack of hardening and the insurgent traffic's tendency to spike with explosive force. Offloading has also proven difficult to accomplish without severe loss. For this reason, if apc use is unavoidable, it is suggested that vital assets remain within it's confines and not be exposed to the enemy network.
posted by IronLizard at 1:34 PM on August 15, 2007

Rewrite the site? Oh, shit, I don't even remember more than a thousand of my posts at the most.
posted by The Deej at 1:35 PM on August 15, 2007 [5 favorites]

We'll call it... porntex.

And seriously, PHP? At least use PHP 5 and use hell of stored procedures all over the place.
posted by boo_radley at 1:43 PM on August 15, 2007

I say we take off, and nuke the site from orbit. It's the only way to be sure.
posted by Cool Papa Bell at 1:49 PM on August 15, 2007 [4 favorites]

cortexAdmin writes "What sort of name is 'Petraeus Haughey'?"

Sounds like a minor character in a Pynchon novel.
posted by brundlefly at 2:11 PM on August 15, 2007

Haughey, you're doing a heckuva job.
posted by dw at 2:14 PM on August 15, 2007

brundleflyPeon writes "Sounds like a minor character in a Pynchon novel."

What would it be called? JRun: a novel?
posted by grouse at 2:17 PM on August 15, 2007

Holy shit. This is possibly the only time in history I've been happy to see a website migrate to PHP. - Plutor

Upgrading from ColdFusion to PHP is like going from being constipated to having diarrhea - sure, it comes out faster, but it's really fucking messy and it usually stinks.
posted by cyrusdogstar at 2:23 PM on August 15, 2007 [3 favorites]

The Crying of User #50830.
posted by goodnewsfortheinsane at 2:25 PM on August 15, 2007

You want faster? What about this Ruby on Rails thing? I heard all the cool kids are using it.
posted by eyeballkid at 2:33 PM on August 15, 2007

Ruby is so last year, all the cool kids use that Javascript full-stack framework now.
posted by cyrusdogstar at 2:35 PM on August 15, 2007

You want faster? What about this Ruby on Rails thing?

That's a joke, right? Please say that is a joke.
posted by grouse at 2:36 PM on August 15, 2007 [1 favorite]

General Haughey? I thought his name was Petraeus.

I hear the JRun insurgency is in its last throes.

Hell, just reinstate the draft!

Jesus Fuckin' Christ, why does every thread around here have to devolve into one about Iraq? WWY2K have to say about this?

/I keed, I keed.
posted by ericb at 2:40 PM on August 15, 2007

Ruby is faster....for developing :)

Actually, now I'm curious how CF stacks up against Ruby or (which I believe is likely to be faster overall than Rails, if not necessarily always up to par with PHP speedwise) Python.
posted by cyrusdogstar at 2:41 PM on August 15, 2007

Fuck You JRun!

*still not over it*
posted by sciurus at 2:43 PM on August 15, 2007

'Good authority' here means the voices in my head.

So, you have a "Committee," too?
posted by ericb at 2:45 PM on August 15, 2007

grouse writes "brundleflyPeon"

Is... is that what you think about me, grouse?

*runs out of thread, crying*
posted by brundlefly at 2:54 PM on August 15, 2007

Jesus Fuckin' Christ, why does every thread around here have to devolve into one about Iraq? WWY2K have to say about this? posted by ericb at 2:40 PM

ericb, please discuss this with pb. He is the Karl Rove architect of the site and originated the Iraq references with talk of "surge".
posted by Cranberry at 2:56 PM on August 15, 2007

I think you should just built a brand new site in parallel with running this one, called metafilter2.com -- duplicate all the user accounts on it, but don't sync posts/comments between the two. Then see how many people are so fed up with the site that they jump over right away, and how many toil in the current blue for various personal reasons.

Then, once you have a good solid base of people on metafilter 2.com, create a point release at metafilter2.1.com and repeat the process ad infinitum. Shrug off the tyranny of archiving past posts! The future is now! Plus, we can repost old links without guilt, as the search function of each new version will not turn up any previous FPPs.
posted by davejay at 3:05 PM on August 15, 2007

of course, you'll have to talk to the people that own 1.com, 2.com, 3.com, etc...
posted by davejay at 3:06 PM on August 15, 2007

That's a joke, right? Please say that is a joke.

I would have thought the "cool kids" comment would have tipped you off.
posted by eyeballkid at 3:34 PM on August 15, 2007

Is... is that what you think about me, grouse?

No, I'm just using a GreaseMonkey extension that adds "Peon" after the name of everyone who isn't mathowie, jessamyn, or cortex. You weren't supposed to see that.
posted by grouse at 3:35 PM on August 15, 2007 [1 favorite]

eyeballkid: Yeah, I thought so, but there are an alarming number of people who think the same way, and aren't joking.
posted by grouse at 3:36 PM on August 15, 2007

No, I'm just using a GreaseMonkey extension that adds "Peon" after the name of everyone who isn't mathowie, jessamyn, or cortex.

I just added pb as an admin in my local MeFi Navigator Greasemonkey script. Now that I've seen it, I think it might need to change to something else.

Codemonkey? The wizard behind the curtain? The invisible fire that works in secret?

Must think on this.
posted by eyeballkid at 3:40 PM on August 15, 2007

I dunno what pb would prefer, but I'm totally stealing 'the invisible fire' for my next business card.
posted by quin at 3:54 PM on August 15, 2007

You'll recognize the Great Rewrite Forward by the posters.
posted by Gary at 4:13 PM on August 15, 2007 [1 favorite]

How about "The Wolf"?
posted by goodnewsfortheinsane at 4:38 PM on August 15, 2007

Nonsense. What sort of name is "Petraeus Haughey"?

Not nearly as cool a name as "Petraeu Shaughey".
posted by oneirodynia at 4:51 PM on August 15, 2007

You know what MeFi needs? A Cool Cam.
posted by BeerFilter at 4:58 PM on August 15, 2007

I like ColdFusion. I've used it for a long time and it's perfectly fine, really. Anyway the truth is it's all about efficient coding, that's all that counts really. Sure CF adds some overhead, but a lot of the intensive stuff can be done with Java objects, and if it can't be done with a Java object it's probobaly overhead that you'd get from any program (database accesses). And while Ruby on Rails may be really easy to start something with, ColdFusion is really easy to maintain (if coded well). You don't need many excess files, there are very few hidden dependencies, and if you want to change how something shows up in the screen you can go right in and change it. Sure it has some typing issues and I've never run it against a super heavy load so who knows what might happen there, but apparently BlueDragon (and possibly the new Railo, which is unrelated to Rails and simply compiles and executes CFML at a breakneck speed) might be a good alternative.

For a lot of the AJAX stuff moving over to CFCs might be a good idea if that hasn't been done already.
posted by Deathalicious at 4:59 PM on August 15, 2007

we've got no business even being in iraqcoldfusion.
posted by quonsar at 5:00 PM on August 15, 2007

Excellent idea. The cool cam would instantly transport you to the biggest flaming firefight occurring at the moment. There's an idea I can get behind.
posted by IronLizard at 5:03 PM on August 15, 2007

I've never run it against a super heavy load so who knows what might happen there,
Who knows indeed?
posted by bonaldi at 5:04 PM on August 15, 2007 [1 favorite]

IronLizard: "Excellent idea. The cool cam would instantly transport you to the biggest flaming firefight occurring at the moment. There's an idea I can get behind."

Actually, I'm sure this could be coded up pretty quickly. The admin interface already shows what's been most-flagged in the past 24 hours. It could be easy to make a link that just redirects there.
posted by Plutor at 5:09 PM on August 15, 2007

We've decided that, in lieu of a Great Rewrite, we'll just make the [more inside] links on the front page of AskMe work as links to the thread proper.

[NOT JOKIST], even. Any bug reports are welcome here.
posted by cortex at 12:18 PM on August 15 [+] [!]

I dunno how much of an improvement this is. It seems redundant and looks busy.
posted by Orrorin at 5:13 PM on August 15, 2007

Just wait a couple of days and you'll forget it changed.
posted by smackfu at 5:28 PM on August 15, 2007

With the added bonus that people will have stopped asking me why in god's name it doesn't do that.
posted by cortex at 5:28 PM on August 15, 2007

Can we still ask jessamyn and mathowie?
posted by Kwine at 5:40 PM on August 15, 2007

This is all so much armchair analysis, but I've always held that the problem with MeFi is the hardware, not the software. It's all still on two boxes (one app, one db), right? MeFi is a very popular site- there are plenty of sites out there doing less with more iron.

Anyway, the logical upgrade path from CF is to JSP.
posted by mkultra at 6:38 PM on August 15, 2007

brundleflyPeon writes "Sounds like a minor character in a Pynchon novel."

What would it be called? JRun: a novel?

Logan's JRun, I should think.
posted by trip and a half at 8:01 PM on August 15, 2007

Let the record show that I was against MeFi's involvement with ColdFusion from the very start.
posted by kyleg at 9:43 PM on August 15, 2007

I agree with Orrorin, and almost posted a MetaTalk to that effect before I noticed this. I'll probably write a Greasemonkey script to kill it in a few days (yeah, I know, overkill - I'm picky).
posted by spaceman_spiff at 9:48 PM on August 15, 2007

hincandenza, i guess you have a point with closed threads. All of them can be served statically and things like faves added etc. can be updated periodically (ie. if i fave a comment in an old thread it's not crucial that the guy who loads it the next second has to see it was just faved.)

Are archived mefi threads cached on disk as webpages or are they generated each time you load them? I realize this is probably not in the cards but staticizing them and then moving them off to a machine running a stripped down web server that just coughs up the HTML/js/images (leaving the server that runs the actual webapp to tend to live threads) can be a huge performance boost.
posted by Firas at 10:34 PM on August 15, 2007

To counter the absurd proposal by the original poster (Complete rewrite?! please...),

I find this confusing and frightening, since it was written by the same person who posted the thread, but I guess it's just me.
posted by stavrosthewonderchicken at 10:45 PM on August 15, 2007

I just assumed the dbl in dblslash stood for double and we just had another MeFite with a split personality Multiple Personality Disorder Dissociative Identity Disorder.

See? I know the correct terminology, even if I don't always use it!
posted by wendell at 11:28 PM on August 15, 2007

We can try out hincandenza's idea right now.

Shame about the cookies.
posted by flabdablet at 6:52 AM on August 16, 2007

I find this confusing and frightening, since it was written by the same person who posted the thread

Poster's remorse.
posted by cortex at 8:50 AM on August 16, 2007

flabdablet: We can try out hincandenza's idea right now.

Shame about the cookies.

That site doesn't work. I'm pretty sure my idea wasn't a non-functional site.
posted by hincandenza at 5:03 PM on August 16, 2007

Works for me.
posted by philomathoholic at 1:59 AM on August 17, 2007

Obviously the secret to Mega Metafilter Performance^tm is to rewrite the whole thing in multithreaded C compiled to custom ASICs in a massively-parallel blade server configuration in conjunction with multiple-homed static content delivery nodes.

I volunteer to...um...paint the server room.

In all seriousness, we've got plenty of coders here on mefi, myself somewhat included. I've always wondered why there's never been a "re-implement mefi" GPL'd project. Then pony requests could be accompanied by a horsie-shaped patch.
posted by Skorgu at 11:15 AM on August 17, 2007

I've always wondered why there's never been a "re-implement mefi" GPL'd project.

There were attempts- Freefilter, MetaPhilter, among others. They're all abandoned, AFAIK.

Because it's much easier to sit and criticize rather than actually do.
posted by mkultra at 8:11 AM on August 19, 2007

I'm not surprised that they were abandoned. Without being sanctioned/blessed by Mr. Haugey, why would anyone work on a niche product that will never be used?

I guess I'm really surprised that #1 isn't substantially lazier.
posted by Skorgu at 8:46 AM on August 20, 2007