Is someone compiling/has someone already compiled statistics?

February 1, 2007 12:40 PM

Is someone compiling/has someone already compiled statistics such as: the average number of favorites per post, the average number of comments per post, comment quantity by time posted, average number of favorited posts by user, etc? I assume this has been discussed here in meta previously, but I couldn't find it in searching. Feel free to direct me to old threads on this topic though.
posted by serazin to Feature Requests at 12:40 PM (40 comments total)

Tread lightly here. A lot of us are pretty annoyed about the existence of favorites statistics. You will hear the phrase "Metafilter is not a popularity contest" more than once in this thread.
posted by koeselitz at 1:08 PM on February 1, 2007

Metafilter is not a... Oh nevermind!
posted by grateful at 1:11 PM on February 1, 2007

Except of course when it is.
posted by It's Raining Florence Henderson at 1:25 PM on February 1, 2007

There have been a few adhoc bits of mefi statistical analysis by various users over the years—see tkolar's recent experiments with AskMe, for example—but I can't recall anything having to do with favorites, specifically.
posted by cortex at 1:26 PM on February 1, 2007

(On the other hand, there's been plenty of discussion of favoriting itself.)
posted by cortex at 1:27 PM on February 1, 2007

Imagine the data we would have if every comment had a quality score!
posted by smackfu at 1:31 PM on February 1, 2007

I like koeselitz' comment more than It's Raining Florence Henderson's, but not as much as cortex's. On the other hand, I don't like this post nearly as much as that one really embarassing medical condition anonymous had that one time. I don't like smackfu's at all, but his username is funny.

So, no, I have no idea what to favorite. Sorry I can't be more help.
posted by ardgedee at 1:35 PM on February 1, 2007 [1 favorite]

To be a little more precise:

It seem to me as though it would be very, very hard to compile statistics on favorites. You'd probably have to look at every user's "favorites" page to have complete picture, and even then you wouldn't know what had been deleted and what hadn't. It's not like stats on number of comments per post or anything; it takes a lot of clicking.

mathowie and jessamyn, it seems, have access to that sort of info inside the big black netherworld that is the brain of Metafilter. (They've used "most favorites" as a deciding factor in a couple contests, anyway.) However, they haven't seen fit to release that information to the public as yet. I don't know if either of them have said anything about this.

So far, this has been an overwhelmingly popular decision. People around here hate the idea of everybody being rated on favorites. This is because every one of us knows, in our heart of hearts, that It's Raining Florence Henderson, that old philandering Roman Senator who continues to fan the flames of his popularity by tossing the ripe corn of his goodly comments to us, the plebians, would win the favorites game by a long shot. We argue endlessly against statistics in the hopes that some young one will arise who, unbeknownst to the rest of us unpopular dictators, is a great favorite of the people, and can defeat Florence and end his unholy reign.

And he will be called the Christ.
posted by koeselitz at 2:00 PM on February 1, 2007

It seem to me as though it would be very, very hard to compile statistics on favorites.

Which actually makes the idea very compelling to me as a personal project. Heh.

(They've used "most favorites" as a deciding factor in a couple contests, anyway.)

Really just a guiding factor, they've said.
posted by cortex at 2:07 PM on February 1, 2007

More than who has the most favorites or ratio of posts to favorites by user or whatever, I'm more curious about how many favorites most posts get.

Scrolling through and manually clicking on each post, it seems that most posts get at least 1 favorite. But most do not seem to get more than 4 or 5. I'd guess maybe 20 - 30% of posts get more than 10 favorites. But I'm just basically guessing here. I'm also curious about whether during certain days of the week, posts are more read, more commented on, or more favorited. Same for time of day.

Sounds like this isn't stuff that's being tracked though.
posted by serazin at 2:09 PM on February 1, 2007

I meant to flag this post but I favorited it by accident. I intended to fix it, but figured if anyone does compile some stats on favorites, my little flub will throw the numbers off.

Hahahaha!! Hahahahah! Hehahehheh! Hermm. Ahemm. **cough, cough**
posted by Terminal Verbosity at 2:12 PM on February 1, 2007

It wouldn't make a difference; your status is determined by your user number. So here are all the stats you need: smackfu is winning this thread so far, and I am 3,273 user-points ahead of you, serazin, while you are nearly 10,000 better than ardgedee. Well done!

As to whether this theory is correct... well, your status denies any possibility of rebutting me, but I would submit to, say, cortex.
posted by hoverboards don't work on water at 2:15 PM on February 1, 2007

but I would submit to, say, cortex

Dammit, I've already said my piece regarding submissal.
posted by cortex at 2:18 PM on February 1, 2007

Ha! Much as the favorites-whore in me loves you for your comment, koeselitz, I'm actually a perfect example of why favorites aren't a good metric for judging anything meaningful here. Unlike jonson, for example (who has way more favorites than I do, anyway), my favorites are based almost entirely on comments, which are mostly of the jokey variety. I post FPPs rarely and unspectacularly. jonson has earned the majority of his favorites by adding vast amounts of actual content to the site. madamjujujive is a great example of someone who has amassed an impressive following in both categories, due to an embarrassment of both wisdom and taste. I'm really more the example of why the statistics alone don't reflect the actual value someone adds to the site.
posted by It's Raining Florence Henderson at 2:21 PM on February 1, 2007 [1 favorite]

This is the most-favorites page that mathowie and I mostly look at, though we can also see the most "fantastic post" flags as well, though the latter seems like it's getting deprecated in favor of the former.
posted by jessamyn (staff) at 2:23 PM on February 1, 2007 [2 favorites]

A lot of us are pretty annoyed about the existence of favorites statistics.

Others would rather "favorites" were instead named "bookmarks" or "memory" or something similar, and that the cheap habit of clicking the plus sign as a means of indicating IAWTP (I Agree With This Post) was discontinued entirely.
posted by cribcage at 2:26 PM on February 1, 2007

And just to clarify, I'm not above wanting to know how awesome everyone thinks I am compared to everyone else, but more than that, I'm trying to figure out what makes a good post.

Since there is no way to know for sure what everyone thinks is a good post, it seems like favorites are one of the best indicators we have (along with comments like, 'good post') of how interesting/useful/exciting/funny whatever the post was to most users.

It's not that I would only post things that I think would get a lot of favorites, but I also don't want to post a bunch of crap that no one wants to read.

Soooo, having a sense of what an average number of favorites is would give me a sense of which posts I've made that people liked better or worse than that average.

To a certain degree this is obvious: a post followed by a lot of interesting discussion, cool further links, and favorites marked is probably a very good post. I was just curious to explore this more, especially wondering if issues like time of post have a significant effect on level of discussion and favorites.

And just to clarify, I'm not really requesting this feature (that was just the catagory most related to my question). I was just curious if someone had been tracking this.
posted by serazin at 2:27 PM on February 1, 2007

I'm really more the example of why the statistics alone don't reflect the actual value someone adds to the site.

Likewise, my favorite count is ridiculously bloated by a couple of mefi-related Music posts; and some folks probably have posts favorited more out of infamy than any positive contribution to the site. It's definitely not something that can be used to say any one specific thing, as interesting as armchair analysis could be.
posted by cortex at 2:29 PM on February 1, 2007

http://www.metafilter.com/favorites/all

That's hugely great - I found a couple of wonderful threads I had somehow missed on the Blue. But as far a fovourites go, I don't use 'em myself, but they can be egobootylicious
posted by Sparx at 3:26 PM on February 1, 2007

'...as favourites...', obviously. dammit, and I'm not even drunk.
posted by Sparx at 3:29 PM on February 1, 2007

I'm more curious about how many favorites most posts get.

Of the 9452 posts in 2006, 4325 were marked as favorites.

5127 posts were marked 0 times
1437 posts were marked 1 time
744 posts were marked 2 times
462 posts were marked 3 times
346 posts were marked 4 times
262 posts were marked 5 times
194 posts were marked 6 times
141 posts were marked 7 times
112 posts were marked 8 times

The remaining 627 posts were marked 9 times or more.

The mean number of favorites was 2.23
The median number of favorites was 0.0
The standard deviation was 6.34
posted by tkolar at 3:42 PM on February 1, 2007

Is that AskMe, specifically, or the site as a whole? (I'm assuming the former, just on volume.)
posted by cortex at 3:46 PM on February 1, 2007

Sorry, that's just metafilter. No AskMe.
posted by tkolar at 3:50 PM on February 1, 2007

Number of comments per post on Metafilter in 2006

Mean: 26.13
Median: 14.00
Standard Deviation: 44.03

Hall of famer: 56002
posted by tkolar at 3:54 PM on February 1, 2007

Oh yeah, here is the original post with all the AskMe stats in it.

I haven't gotten around to doing hourly based comment graphs, but this one of daily average answers per question illustrates the weekly pattern fairly well. Each one of those cycles is exactly one week long.
posted by tkolar at 4:03 PM on February 1, 2007

"Metafilter is not a popularity contest"

Well, that's a let-down!... And here I've been, working for hours each day perfecting a "Please Like Me" tap dancing routine, complete with peekaboo costume, flaming baton AND sparklers, that I was going to post in Projects. Now it's all for naught.
posted by amyms at 4:20 PM on February 1, 2007

tkolar - this is EXACTLY what I was looking for! You are hella fresh! Thank you.
posted by serazin at 4:24 PM on February 1, 2007

Others would rather "favorites" were instead named "bookmarks" or "memory" or something similar, and that the cheap habit of clicking the plus sign as a means of indicating IAWTP (I Agree With This Post) was discontinued entirely.

Yep, I agree (and almost favorited your comment, in an ironic way).

I've noticed that that trend seems to be on the rise, which was predictable and predicted. It's interesting to watch the way it all plays out, actually -- we have in effect a one-directional two-value voting system, like a rose-coloured-glasses version of slashdot or digg, where people register their approval of whatever strikes their fancy (if that's the way they use the system), but have no way of registering their disapproval, other than actually typing up a comment.

I wonder if that greater visibility of negative commentary means a Larger Visible Snark Quotient. Interesting...

I wonder if this visibility of up-voting only was by design, and if it was, I once again tip my hat to Matt, as I think it's a pretty damn good way of introducing positive feedback for quality participation without turning it too much into the 'modded down LOL YOU SUX0R!' shitstorm that you can see at those other sites, where down-voted comments below a threshold are collapsed. The flat nature of MeFi threads keeps that from being a possibility. There are, as always, weird and wonderful unforseen consequences, though.

I've argued at length in the past that invisible downmodding (through the flagging system) was pernicious, and I still think it's a bit dodgy and non-scalable, but the gestalt output of the new systems' interactions has had intriguing consequences, and is continuing to evolve.

Also, again, I too wish that Matt had chosen another word besides 'favorites', but again again, if it's been a matter of Community Steering By Design, then that choice makes perfect sense.

Unfortunately, having listened to the interview with him on the boingboing podcast, I'm wondering if it was more a matter of 'well, I'll toss this in the mix and see'. Who knows -- me, I wish he released full-on design documents and philosopical treatises on web community and stuff, rather than making us do all the heavy beard-strokery.
posted by stavrosthewonderchicken at 4:30 PM on February 1, 2007

"Metafilter is not a popularity contest"

Yes it is! and I've already won!

*preens before throng who then tear him to pieces*
posted by jonmc at 4:40 PM on February 1, 2007

/Hold up a piece of jonmc

Anyone got any A1 sauce?

Also thanks for that link jessamyn.
posted by quin at 5:44 PM on February 1, 2007

stavros, I think you probably overestimate the impact of flags and favorites on the site. Ultimately, nothing happens unless matt or jessamyn does it.
posted by empath at 6:01 PM on February 1, 2007

stavros, I think you probably overestimate the impact of flags and favorites on the site.

Not at all. I've been closely watching the ebb and flow of the 'community' aspect of the site for years, and spent way too much time thinking about it -- it's probably my primary interest in MeFi, and why I spend more time in Metatalk than anywhere else. Even small changes (two of which have been flags and faves) have had quite significant impacts on overall tenor and tone, sometimes deliberate, often not. It's clearly subjective, and certainly the triggers of sea change have been kicked off by admin and design decisions, but the subsequent courses we've floated along as a result of the million little interactions between the thousands of users here every day have been charted and initiated but not navigated by Matt and Jess (if that makes sense, and to stretch the metaphor to the breaking point).

Ultimately, nothing happens unless matt or jessamyn does it.

No, again. Matt and Jess (god bless 'em) nudge the tiller; it is the thousands of regular users that are the wind. No wind, and the tiller does nothing.
posted by stavrosthewonderchicken at 6:13 PM on February 1, 2007

I think you probably overestimate the impact of flags and favorites on the site.

No, I think he's right. I think the flags are used akin to moderation — and specifically, I think they're overwhelmingly used (on the blue) to indicate agreement with exactly the type of low-content, high-sarcasm snark that drops the level of discourse. Those comments suck and MetaFilter needs fewer of them, and I think the primary impact of "favorites" has been to endorse them. That's bad.

Put another way: It's bad enough when a new user comes along and sees an interesting thread riddled with rude, childish insults. That's Fark. But it's worse when every one of those insults is tagged with, "This comment was endorsed by 6 members of the MetaFilter community." It makes us all look bad, like we're cheering for the wrong stuff.

The comparison to Slashdot is interesting, because in my experience those type of quips rarely get modded +4 or +5. That's where I usually read*; and in fact, when I have occasionally dropped below 3, I've almost never found an insightful, deserving comment that wasn't recognized by the moderators. Maybe it's an artifact of anonymous moderation, or maybe Slashdot members are smarter than MeFites.

* If you're not familiar with Slashdot: Comments are scored between 0–5, and you can adjust your preferences so that comments below a certain score are hidden from view. I usually read at +4, which means that anything scored between 0–3 is hidden.
posted by cribcage at 7:48 PM on February 1, 2007

I've often wondered how many times an average FPP gets clicked through and read, and whether there is any correlation between the number of comments an FPP gets and how many times it gets read.
posted by alms at 7:57 PM on February 1, 2007

Put another way: It's bad enough when a new user comes along and sees an interesting thread riddled with rude, childish insults. That's Fark. But it's worse when every one of those insults is tagged with, "This comment was endorsed by 6 members of the MetaFilter community." It makes us all look bad, like we're cheering for the wrong stuff.

That's not at all what I was saying, but it's an interesting and valid point.
posted by stavrosthewonderchicken at 8:12 PM on February 1, 2007

I think they're overwhelmingly used (on the blue) to indicate agreement with exactly the type of low-content, high-sarcasm snark that drops the level of discourse.

That's an interesting thought, but it's completely at odds with the top favorites page pointed to earlier by jessamyn.
posted by tkolar at 8:35 PM on February 1, 2007

it's completely at odds with the top favorites page

No, that's a different statistic. There's no reason to think the most-favorited posts are necessarily indicative of how MeFites generally use the feature. I'd wager that, for every brilliant post that attracts >30 favorites, there are at least a dozen trollish snarks that attract between 2–6.

On the other hand, your point does contradict my attempt to contrast MetaFilter with Slashdot. That page would seem to indicate that our makeshift moderation system does a decent job, past a certain level, of spotlighting the best contributions. Maybe viewing that page is like the difference between reading Slashdot at +4 versus +2.
posted by cribcage at 8:56 PM on February 1, 2007

I'd wager that, for every brilliant post that attracts >30 favorites, there are at least a dozen trollish snarks that attract between 2–6.

Unfortunately this is a very difficult statement to prove or disprove.

My instincts disagree with yours on the matter, but if there's one thing that the last month of metafilter statistics has taught me it's that nobody really has a really solid grip on what goes on around here.
posted by tkolar at 9:33 PM on February 1, 2007

tkolar - looking over the posts from today, almost all had at least one favorite. Maybe it was a 'good' day around here, but that average is a lot different than last year's overall average. I wonder if things are changing or if today was a randomly high in favorites day.
posted by serazin at 9:48 PM on February 1, 2007

People use the favouriting system in different ways so stats will always be a difficult to interpret. If you took out comments with less than say 4 faves I would think you would be rid of the great majority of snarksupport. But I sometimes like to see who has clicked the '+' for whatever reason; it can be an interesting way to 'read' the community and individual tastes/sensiblities.

One metric I think is good - when you see a post with more faves than comments. There is usually stellar quality material at the end of the click supporting the notion that this site works best when it's about the links rather than the discussion (I like both btw).
posted by peacay at 10:08 PM on February 1, 2007