Has anyone thought of grabbing a snapshot...?

March 12, 2006 6:07 PM

Has anyone thought of grabbing a snapshot of pages linked from MetaFilter, a la Yahoo MyWeb?
posted by MarkO to MetaFilter-Related at 6:07 PM (14 comments total)

Cuz if you were to search the MeFi archives, find an interesting yet cryptic post, and then click the link only to discover that it's a 404 or a link farm, wouldn't you be bummed? I guess there's archive.org, but that's not perfect.
posted by MarkO at 6:09 PM on March 12, 2006

no, nobody has thought of that. next question?
posted by quonsar at 6:15 PM on March 12, 2006

What does the q stand for in quonsar?
posted by TwelveTwo at 6:19 PM on March 12, 2006

quanky? qwusty?
posted by popechunk at 6:34 PM on March 12, 2006

quasshole?

You mean a cache of everything ever linked? That would be difficult and problematic to store and retreive, not to mention possible copyright concerns on the part of site owners that get cached (and they don't get ad revenue when their pages are loaded from this site).
posted by mathowie (staff) at 6:45 PM on March 12, 2006

That's kind of the web in a nutshell isn't it? That's just how it goes... I think the server resources (not to mention copyright issues) needed to support such a project would be gigantic.
posted by Rhomboid at 6:47 PM on March 12, 2006

LOL@Matt
posted by Balisong at 7:00 PM on March 12, 2006

I can't see the point of trying to replicate archive.org if that's what you mean.
posted by keijo at 7:44 PM on March 12, 2006

one day, someone's (or some team of ppl) going to have to go thru all the old posts and either flag em if they're dead, or change the links to go to archive.org, no?
posted by amberglow at 7:55 PM on March 12, 2006

bah! you revisionist, you!
posted by crunchland at 7:58 PM on March 12, 2006

one day, someone's (or some team of ppl) going to have to go thru all the old posts and either flag em if they're dead, or change the links to go to archive.org, no?
No.
posted by dg at 8:31 PM on March 12, 2006

You could write a clever little script to test for (1) unresponsive domain, (2) 404s, (3) apparently unrelated content (based on some, I dunno, bayesian hash of the thread versus the current content of the target page), and provide an alternative archive.org link automatically.

Implementation left as an exercise for the reader.
posted by cortex at 10:04 PM on March 12, 2006

I had the same thought as cortex. The problem is that archive's URLs are dependant on exactly the date it was backed up. You'd have to search through their archive list for a given page and link to the closest one. Which would be Righter, the last cache before the MeFi link, or the first cache after it?
posted by Plutor at 7:14 AM on March 13, 2006

the first after, i'd say...that way we get whatever response occurred.
posted by amberglow at 9:40 AM on March 13, 2006