Export the full text of Metafilter favorites

July 9, 2014 6:08 AM

I'd like to export the "wisdom of Metafilter" that I've favorited over the years, but I'd like to do it with the least amount of effort and risk of error. The two methods I've found so far will require quite a bit of copy/pasting. Am I missing a special killer-combo-move, or will I just have to roll up my sleeves and get my ctrl+ C/P fingers warmed up?

I have over 600 favorites that I'd like to export so I can edit and rearrange them in a scrapbook-y project. The two methods of exporting favorited comments I've found so far would require quite a bit of copy/pasting in order to complete the project:

- Export All Favorites - Which gives you HTML links to the favorite in the thread
- View Favorites - Which is great, but sometimes gives you a truncated >more

Color me lazy, but is there any way to export the full text of my favorites so I can limit copy pasting and not screw it up by missing comments?

posted by guy72277 to MetaFilter-Related at 6:08 AM (13 comments total)

No, sorry, we don't have a built-in way to bulk export the text of other people's comments. (You can export the full text of your own comments via the Preferences page.)

The list of URLs will give you the permanent home of the comments so you can read them on the site. We're not planning to go anywhere so the comments should always be available online. And if we do go somewhere the Internet Archive should have a backup of those comments saved that the URL should bring up.

As you noted, you can read fairly long excerpts of the comments on your Favorites page. You can also search through your favorites by keyword.
posted by pb (staff) at 6:54 AM on July 9, 2014

Is this practice for the 20th anniversary, when we'll be printing up our own individual comments before the big Westphalia meetup where we'll then recreate MetaFilter by arranging them all in a field in correct order? Thought so.
posted by heyho at 7:46 AM on July 9, 2014 [6 favorites]

So it's patience and diligence in large amounts then.... Thanks for the info pb.

Heyho - I can neither confirm nor deny your speculation.
posted by guy72277 at 8:24 AM on July 9, 2014

One not-lazy automated approach you could take is to grab your Export and then feed that into a small scraper script that would request the appropriate thread for each favorited item, isolate the item in the retrieved page, and then chuck that out to a new location according to your needs.

Scraping content off a semi-arbitrary page can be an error-prone pain to set up and then test, though, so whether that was actually worth the time vs. a one-off copy-paste session is a pretty reasonable question if you don't already have the whole skillset in your back pocket.
posted by cortex (staff) at 8:39 AM on July 9, 2014 [1 favorite]

I've asked for something similar before. I can never find comments I've faved through the search function because I can never guess at the right words in the comment - I can only remember the theme.
posted by triggerfinger at 10:13 AM on July 9, 2014

i had no idea! Awesome! Check out this sweet wordcloud i made of all my comments
posted by rebent at 12:16 PM on July 9, 2014

Check out this sweet wordcloud

Metafilter: people really like think
posted by postcommunism at 1:37 PM on July 9, 2014 [2 favorites]

Check out this sweet wordcloud

Good idea! This seems pretty much right I guess.
posted by cortex (staff) at 1:54 PM on July 9, 2014 [4 favorites]

Here's a Python script that might help. It requires the beautifulsoup4 library. Sorry if it's kind of messy; let me know if you have any questions.
posted by caaaaaam at 3:02 PM on July 9, 2014 [1 favorite]

cortex 'n' caaaaaam

Thanks for your input. I'll look into scraping with the python script but I agree that I'll maybe spend more time figuring it out and checking it than I would just biting the bullet and doing it manually.

Summer's here and work is slowing down so it could be a good time to tackle this problem (and the 750 'interesting comments' from around the internet that I've emailed to Gmail over the years).
posted by guy72277 at 12:45 AM on July 10, 2014

This is something that I might hire a mechanical turk to do for me.
posted by klangklangston at 12:34 PM on July 10, 2014

I'll maybe spend more time figuring it out and checking it than I would just biting the bullet and doing it manually

If you wanna just send me your favorites export file, I can easily run this for you in about 5 minutes.
posted by caaaaaam at 1:13 PM on July 10, 2014

caaaaaam that is a really generous offer and I'm very happy to accept it.

I'll carry this over to a pm then and work out how best to send my favorites export file to you.
posted by guy72277 at 12:57 AM on July 11, 2014