Nerds, start your engines: it's the new Metafilter Infodump
posted by cortex to MetaFilter-Related at 7:58 AM (279 comments total)
48 users marked this as a favorite
There's been a lot of requests over the years for crunchy metafilter data (and another spike of requests lately). This is an attempt to make some of that available in a standard form in a reliable, officially sanctioned location. This is a compromise between the fantasy (a full db dump, or a live API for db requests) and the previous reality (nothing!); it has the advantage of actually existing, which I think is a big step up.
Hopefully, this will save enterprising number crunchers some scraping, and lower the bar for entry some for those folks who would like to play with data but don't want to scrape.
All of the files are plain text. Some are quite large. I've listed approximate file sizes for each file. They're not updated automatically at this point, but I can regenerate them as needed, and will if nothing else run the scripts once a month or so.
If there's something specific you'd like to see added to or tweaked on the Infodump, let me know, here or via mail. If you do something cool with the data (or have done cool things with mefi data in the past), also let me know; I'd love to put together a permanent collection of mefi analyses.