Where is my pony? 0 answers [100 favorites]

May 5, 2009 10:59 AM

Is there compelling data in "Popular Favorites"?

I really like perusing ask.mefi on the "popular favorites" tab. I usually see what's on the front page and then if I have time use "favorites" to go a bit deeper and see what I've missed over the last day or so. What I find amusing is when there is a question with maybe a couple answers but a huge amount of favorites. What that says to me is that here's a question that a lot of people are interested in but not many people know of a good answer. And I feel like there's compelling data there... but what is it? Is there any compelling social data that can come out of there? And will one of you smart people pull it out and present it to the rest of us? That's my pony!

posted by amanda to MetaFilter-related at 10:59 AM (21 comments total)

Somewhere out there, cortex is quivering.
posted by It's Raining Florence Henderson at 11:05 AM on May 5 [1 favorite]

I think there's probably a few different things going on with heavily-favorited questions, and some separate things going on with sparsely-answered questions, and like you say there's probably some compelling information about how people use AskMe to be found by looking at those two things and where they overlap.

The biggest favorites-multiplier, by my gut interpretation, is what I think of as "referencability": questions that promise some significant degree of general utility in the likely answers, that in other words promise (at asking time) or manifest (after answers have already flowed in) usefulness as a reference for some sort of information. High referencability means a lot of people can see some potential utility in coming back to that question as a reference, which means a lot of people favorite it as a bookmarking method.

But referencability doesn't really explain heavily favorited questions that produce few answers—a highly referenceable thread would usually have a lot of answers, you'd think. There's plausible exceptions—a question that contains in its text a novel and referencable link or set of links, or a question whose few answers are really useful (though a single really referencable answer would probably get the favorites itself, I'd think), or a question that represents a really compelling mystery that folks hope to come back to later and find solved—but in general it probably doesn't fit.

So it's an interesting question, yeah.

Here's a great big pile of data (it's a mirror of our currently-down Infodump feature) that should make sorting through the askme archives a lot more practical than just sifting the Popular Favorites page, if anyone feels like crunching.

I'd love to see a scatterplot of comments vs. favorites, actually—how are threads distributed across that plane? If the points are plotted by category, is there any significant grouping in space?
posted by cortex at 11:10 AM on May 5 [2 favorites]

It might be interesting to try it with MeFi too, since a lot of posts on the blue that are really good are so good that they don't require much additional discussion--just a lot of "WOW!"--and so don't garner a lot of comments, but get teh fayvritz.

Also--Big, contentious threads: Lots of favorites?
posted by Sys Rq at 11:20 AM on May 5

Here's the trend line:

|----------------------------------------------------------------------------------------------------|

I did some linear regression on it, but must have miscalculated, because I keep getting a straight line.
posted by blue_beetle at 11:22 AM on May 5 [1 favorite]

Also--Big, contentious threads: Lots of favorites?

Usually not because of the bigness and the contention, I think. If they get a lot of favorites, it's because of content, either in the post itself or later in the discussion. I think there are probably a lot more favorites on individual comments in those threads rather than on the main post itself.
posted by ocherdraco at 11:24 AM on May 5

And a couple thoughts for potential data divers:

- Favorites didn't exist until May 10th, 2006, so any analysis of favoriting behavior needs to take that into account. AskMe threads posted before that date should probably be excluded from any general analysis that uses favorites as presumed-normal metric, and really even stuff after launch date for a while is kind of suspect since rate-of-favoriting took a while to ramp up and level out if I remember some previous experiments correctly.

- Speaking of which, a thorough analysis of favoriting vs. time would be a pretty interesting thing to see.

- And so would an analysis of late favoriting—favorites attached to something after some interval of immediacy (three days? a week? a month?) has passed. For example, pretty much anything posted before May 10th, 2006 had to have been favorited late if it was favorited at all. What does favoriting behavior only after-the-fact look like compared to total aggregate favoriting, and how evenly distributed is it over time? Etc.
posted by cortex at 11:26 AM on May 5

Somewhere out there, cortex is quivering.

TMI
posted by dersins at 11:38 AM on May 5

cortex: - Favorites didn't exist until May 10th, 2006, so any analysis of favoriting behavior needs to take that into account.

It would probably be prudent to start the analysis even later than that because it took a while for people to get used to this feature. It certainly did for me. I'd pick January 1st, 2007 as a nicely arbitrary starting date while still catching a large enough sample of AskMe posts.
posted by Kattullus at 11:39 AM on May 5

The power of Christ compels your data.
posted by crapmatic at 12:07 PM on May 5

Ah, the bookmarking aspect of it is pretty interesting (and could be monetized?*). That's how I use "favorites" in terms of a thread. I favorite a post so that I can come back later and either review the collected data (comments) or because all the data in it is good (great links, etc.). I favorite comments to essentially "+1" the info if it is saying almost entirely what I would say in a thread. Or if someone says something super witty. I like to reward the super witty.

I think this has been answered elsewhere but the threads with favorites on "popular favorites" reflect people favoriting the entire post, not just the ensuing comments, correct?

* So, even though I'm interested in the social data concepts of "popular favorites" I think it would be pretty interesting to at least plot the favorited posts in regards to category. So, a home&garden post with tons of favorites which talks specifically about building a foundation might have sponsor links that are relevant. Perhaps even for a paid subscriber, after a period of time, old posts like that which are favorited get sponsor links. After all, mefi is my handy link bookmarker of a certain sort so it would be my way of paying back.

BUT, this is getting afield because I have a little evil-marketing portion of my brain which is hard to turn off.

It would be interesting to see trends in regards to time -- some questions are definitely topical but they also reflect maybe larger societal trends -- growing your own garden, interest in renewable energy, questions about performance-enhancing drugs. If you could correspond even 1/4 of the data with news sources, that might be interesting. Or... not?

Thanks for indulging, cortex!
posted by amanda at 12:22 PM on May 5 [1 favorite]

cortex: I'd love to see a scatterplot of comments vs. favorites, actually—how are threads distributed across that plane? If the points are plotted by category, is there any significant grouping in space?

I've had a tab open on the wiki page of MetaAnalysis (MetaAnalyses?) for a while now, thinking about categorizing the entries there. So: graphs of favorites vs. comments. And, categories weren't / aren't included in the infodump.
posted by Pronoiac at 12:24 PM on May 5

I've had a tab open

Oh, shit, yeah. And, yeah, categories not in the dump, right. No wonder this felt familiar.

I think this has been answered elsewhere but the threads with favorites on "popular favorites" reflect people favoriting the entire post, not just the ensuing comments, correct?

Correct. It's only the favorites on a given item (be it post or comment) that gets that item on the listing.
posted by cortex at 12:43 PM on May 5

An example from the current "popular favorites" tab:

I'm having a really hard time dealing with news of humans' brutality to other humans. When I read the news (esp. that coming out of the African continent) , my immediate response is that, as a species, we simply don't deserve to exist. I need some recommendations for literature, films, poetry, that addresses these issues and comes out optimistic. No Cormac McCarthy thanks.
[more inside]
posted by xenoworx at 9:23 AM May 04 2009
57 answers [37 favorites (23 in the last 24 hours)]

So, there's some essential truth and interest just to this data. This is one of those questions which I suspect in real life would garner a "yeah, life's tough" response but here 57 people have felt compelled to respond and near 40 want to bookmark this for later. Am I just a squishy liberal with a soft spot for pathos or is there something more interesting here? I suppose the only data that could be gleaned on that topic is best left to social anthropologists.

Anyway....
posted by amanda at 12:43 PM on May 5

I don't know. Is our children learning?
posted by fourcheesemac at 12:44 PM on May 5

Another thing that could be interesting: taking the tags associated with posts and plotting those, in aggregate, on the scatter of comments vs. favorites in the same sense as the proposed category-as-third-dimension idea.
posted by cortex at 12:44 PM on May 5

Well, my take on the xenoworx example is that there's one very, very key clause in there that explains both the pile of answers and the favorites (coming back to referencability, here):

I need some recommendations for literature, films, poetry, that addresses...

People (a) like giving recommendations about arts and literature and (b) enjoy mild constraints when they allow one to answer with even a mild sense of expert specificity, and people also like (c) reading through or being able to come back to a list of suggestions.

So my take is that regardless of whether there's any measurable pathos/zeitgeist-as-motivator effect at play (something I think would be hard to test for, because how do you A/B this sort of thing? where do you get a control group, and how do you eliminate or correct for noise?), it would be dwarfed by the much more straightforward people-enjoy-this-form-of-thread effect.
posted by cortex at 12:50 PM on May 5

Somewhere out there, cortex is quivering. In my pants.
posted by loquacious at 1:01 PM on May 5

The xenoworx AskMe is also somewhat skewed by the fact that there was a MeTa posted about it. Not that the AskMe itself was particularly contentious, but the MeTa probably drew more people to that thread.

There's another good one! FPPs (wince) and AskMes with their favorites before and after they've been called out in MeTa!
posted by SpiffyRob at 1:19 PM on May 5

cortex: Another thing that could be interesting: taking the tags associated with posts and plotting those, in aggregate, on the scatter of comments vs. favorites in the same sense as the proposed category-as-third-dimension idea.

There's some matrix decomposition something something tutorial going on with the Netflix Prize. I'll look for it later.

Somebody check out the MetaAnalysis page for clarity, plz?
posted by Pronoiac at 8:09 PM on May 5

I looked at this a couple months ago.

The extreme high favorites/comment were bookmarking posts, like the short-form animation post, and the how to cook Indian food post.

Things change, but when I ran this, the most favs per comment
posted by rakish_yet_centered at 8:16 PM on May 5

You accidentally part of the sentence, rakish_yet_centered.
posted by Pronoiac at 9:07 PM on May 5