Comment milestones July 11, 2004 9:14 AM Subscribe
Comment milestones over time. Some comment data for stats geeks inside.
I think it's probably because the number of active, commenting users is relatively stable over time, due to factors like registration lockdown and people generally drifting away from the site after a while. Give it long enough without opening up new registrations - or alternatively, opening up new registrations with restrictions for an extended period - and that curve will change.
posted by adrianhon at 9:34 AM on July 11, 2004
posted by adrianhon at 9:34 AM on July 11, 2004
Isn't it that the number of steady/regular users has always stayed pretty constant? (due to older members dropping out, and that most people rarely comment or post, etc?)
or, what adrianhon said. : >
posted by amberglow at 9:34 AM on July 11, 2004
or, what adrianhon said. : >
posted by amberglow at 9:34 AM on July 11, 2004
It would be interesting to see the number of FPP charted in the same manner. Is this straight line correlated to the number of posts or is the number of comments per post changing?
What does Meta Talk look like over the same period, is the comment variability being sucked off the front page to here.
This being a closed system during most of that time the statistical makeup of the heavy commenters to the occasional commenter, like myself, should remain steady and if the number of posts remains steady then I would expect the straight line effect.
Or, there is only so much that can be said about anything at any given time.
posted by mss at 9:36 AM on July 11, 2004
What does Meta Talk look like over the same period, is the comment variability being sucked off the front page to here.
This being a closed system during most of that time the statistical makeup of the heavy commenters to the occasional commenter, like myself, should remain steady and if the number of posts remains steady then I would expect the straight line effect.
Or, there is only so much that can be said about anything at any given time.
posted by mss at 9:36 AM on July 11, 2004
That's pretty cool.
posted by lazaruslong at 9:41 AM on July 11, 2004
posted by lazaruslong at 9:41 AM on July 11, 2004
oops, I meant, 'opening up new registrations without restrictions'.
mss: I know that I've basically given up commenting on Mefi because generally what I want to say has already been said by someone else, and I don't want to get drawn into a flamewar. As a result I have taken myself and my comments to greener climes...
posted by adrianhon at 9:43 AM on July 11, 2004
mss: I know that I've basically given up commenting on Mefi because generally what I want to say has already been said by someone else, and I don't want to get drawn into a flamewar. As a result I have taken myself and my comments to greener climes...
posted by adrianhon at 9:43 AM on July 11, 2004
I wonder if there's kind of an organic limit where people feel like a thread is 'full enough' and don't bother joining in the conversation...
posted by Space Coyote at 9:53 AM on July 11, 2004
posted by Space Coyote at 9:53 AM on July 11, 2004
There's a question I've always wondered that, if you can answer it, will also answer matt's question too:
How is it, with so many users, each posting FPPs only occassionally at almost random intervals with no advance knowledge of who else is going to post an FPP that day, the front page has never had a say with only a few posts or alternatively hundreds of posts? Its always about the same even though the posters themselves are an unpredictable lot - sometimes getting people who have only posted a few times here.
I think there's a law of averages at play combined with an expectation of what the front page should look like. Also,
people dont like to comment on older threads where their comments might not be seen. Since new posts scroll down the front page at a fairly steady rate (see above), posts also "age" at this rate.
My local gourmet grocery store has hundreds and hundreds of clients. But when I go shop there, there is always the same number of other people shopping - usually 10-15. Its a different 10-15 people each time and yet the number stays steady. This has always fascinated me.
posted by vacapinta at 10:05 AM on July 11, 2004
How is it, with so many users, each posting FPPs only occassionally at almost random intervals with no advance knowledge of who else is going to post an FPP that day, the front page has never had a say with only a few posts or alternatively hundreds of posts? Its always about the same even though the posters themselves are an unpredictable lot - sometimes getting people who have only posted a few times here.
I think there's a law of averages at play combined with an expectation of what the front page should look like. Also,
people dont like to comment on older threads where their comments might not be seen. Since new posts scroll down the front page at a fairly steady rate (see above), posts also "age" at this rate.
My local gourmet grocery store has hundreds and hundreds of clients. But when I go shop there, there is always the same number of other people shopping - usually 10-15. Its a different 10-15 people each time and yet the number stays steady. This has always fascinated me.
posted by vacapinta at 10:05 AM on July 11, 2004
I answered one of my own questions using Waxy data although its time based instead of comment milestone based. It shows a straight line rise until about Aug 02 and since then it has stayed at 26 comments per post + - 2ish
posted by mss at 10:07 AM on July 11, 2004
posted by mss at 10:07 AM on July 11, 2004
There are lots of technical changes you could make to the site that would add a curve to that graph for the short term. Threaded conversations and redesign the front page to keep articles up front for a longer period for example. But most would still resume the R2=1 line after a period of adjustment. Only opening up registration and making the site a free for all would give the site a sustained curve on that graph.
posted by Voivod at 10:20 AM on July 11, 2004
posted by Voivod at 10:20 AM on July 11, 2004
yeah, mss, good data to compare against this data would be number of threads/day and thread milestones over time.
posted by mathowie (staff) at 10:21 AM on July 11, 2004
posted by mathowie (staff) at 10:21 AM on July 11, 2004
Now you're going to have to watch how/when you open user registrations for fear of screwing up the perfect line.
posted by ODiV at 11:02 AM on July 11, 2004
posted by ODiV at 11:02 AM on July 11, 2004
Speaking to what Space Coyote said, I personally won't post in a thread if there are more then 50 or so messages -- even if I have something to say.
posted by chunking express at 11:05 AM on July 11, 2004
posted by chunking express at 11:05 AM on July 11, 2004
If you were to include AxMe posts in this and I think the numbers would jump over the last 6 months.
posted by falconred at 11:09 AM on July 11, 2004
posted by falconred at 11:09 AM on July 11, 2004
Ask MetaFilter has only been around for 6 months, so longterm growth isn't really worth watching there yet.
And just to make it clear, I'm not interested in there being more comments or less comments, just that I noticed an odd trend that I felt like sharing -- that even during changes to the site, one major data point stayed steady.
posted by mathowie (staff) at 11:16 AM on July 11, 2004
And just to make it clear, I'm not interested in there being more comments or less comments, just that I noticed an odd trend that I felt like sharing -- that even during changes to the site, one major data point stayed steady.
posted by mathowie (staff) at 11:16 AM on July 11, 2004
It would be interesting to look more closely and see if there are meaningful internal bounds within the overall behavior you've spotted. I can think of two distributions that might be interesting to plot:
1) Put some analysis behind vacapinta's point, and look at the distribution of FPPs/day. (How many days have had 1-5 FPPs, how many days had 6-10 FPPs, etc.)
2) Similarly, look at the SpaceCoyote's point at the comment lengths of threads, and see how the volume of response per FPP is distributed. (How many FPPs generated 0 comments, how many FPPs generated discussions of 1-10 comments, 11-20 comments, etc?) (To second or third his point, if a discussion's gone past 70 or so comments, I really don't think much about commenting--the odds of it having degenerated into some kind of flamewar/callout spat approaches 100%.)
Put together, those points frame the hypothesis that if you've got a robust audience that will throw FPPs and comments into the daily pool until it's full, but there are commonly accepted definitions of "full", then you're going to get very consistent behavior like you've found over time. Given the fact that people seem to "self-bound" the number of FPPs per day, in practice, then if they also "self-bounded" the number of comments per FPP, it would make sense that the overall number of comments would be closely bounded as well. In fact, over time, the two factors would probably "even each other out", statistically, and contribute to a more consistent overall trend.
Just a thought.
posted by LairBob at 11:50 AM on July 11, 2004
1) Put some analysis behind vacapinta's point, and look at the distribution of FPPs/day. (How many days have had 1-5 FPPs, how many days had 6-10 FPPs, etc.)
2) Similarly, look at the SpaceCoyote's point at the comment lengths of threads, and see how the volume of response per FPP is distributed. (How many FPPs generated 0 comments, how many FPPs generated discussions of 1-10 comments, 11-20 comments, etc?) (To second or third his point, if a discussion's gone past 70 or so comments, I really don't think much about commenting--the odds of it having degenerated into some kind of flamewar/callout spat approaches 100%.)
Put together, those points frame the hypothesis that if you've got a robust audience that will throw FPPs and comments into the daily pool until it's full, but there are commonly accepted definitions of "full", then you're going to get very consistent behavior like you've found over time. Given the fact that people seem to "self-bound" the number of FPPs per day, in practice, then if they also "self-bounded" the number of comments per FPP, it would make sense that the overall number of comments would be closely bounded as well. In fact, over time, the two factors would probably "even each other out", statistically, and contribute to a more consistent overall trend.
Just a thought.
posted by LairBob at 11:50 AM on July 11, 2004
Perhaps I'm missing something, but there haven't been that many new members over the past couple of years (17000-17500 almost spans two years).. yet comments have climbed steadily. This must mean that the average MeFier is contributing a lot more than ever before. I wonder if this is due to the media exposure this site has been getting.
posted by wackybrit at 11:52 AM on July 11, 2004
posted by wackybrit at 11:52 AM on July 11, 2004
I just worked out I now have 1500 total comments, so I've contributed 0.21% of all the comments in MeFiLand, and I'm a freakin' lightweight :-)
posted by wackybrit at 11:54 AM on July 11, 2004
posted by wackybrit at 11:54 AM on July 11, 2004
wacky, given that the number of users has actually risen over that time, there are actually fewer comments per member now than there were three years ago. (200K comments/year for X users in 2001, and now 200K comments/year for X+Y users in 2004.)
That's not to say, though, that the average active member is posting more or less. It might also be interesting to look at the number of "active" posters over time--a threshold of X comments per month, maybe. If you could establish a useful threshold to define "active" members, it might be meaningful to see if that number has changed as membership has grown, or whether the number of active participants has held steady with the rate of commenting.
[That's the last suggestion for now, promise.]
posted by LairBob at 12:20 PM on July 11, 2004
That's not to say, though, that the average active member is posting more or less. It might also be interesting to look at the number of "active" posters over time--a threshold of X comments per month, maybe. If you could establish a useful threshold to define "active" members, it might be meaningful to see if that number has changed as membership has grown, or whether the number of active participants has held steady with the rate of commenting.
[That's the last suggestion for now, promise.]
posted by LairBob at 12:20 PM on July 11, 2004
Maybe it is the invisible hand of God.
posted by Keyser Soze at 12:36 PM on July 11, 2004
posted by Keyser Soze at 12:36 PM on July 11, 2004
The ultimate statistical analysis would be the distribution of comments or posts among users; how many remain silent (or almost so) vs. how many of us hit the keyboard at the slightest provocation. And how many of us make meaningless comments in a thread just so it will show up on our own userpage lists for future reference.
Not that i'm doing that... at least not right here.
posted by wendell at 1:11 PM on July 11, 2004
Not that i'm doing that... at least not right here.
posted by wendell at 1:11 PM on July 11, 2004
The reason that the comment count has increased at a steady rate despite user numbers staying relatively static could be that, with membership being mostly closed, the likelihood of someone signing up just because they can and it is easy is fairly small. New users tend to be those who have been lurking for a while and jump on the chance to sign up when they can. If sign-ups were always open, you would probably see the increase in comments pegged to the sign-up of new users. Perhaps, because membership is hard to get, it is valued highly and those who get in tend to stick around and participate because they value the membership.
I have noticed that the number of new threads seems to be fairly constant (with less on US weekends) and suspect that, once the number of threads on a given day reaches x, people hold off for fear of their thread being overlooked.
Interesting stuff.
posted by dg at 3:54 PM on July 11, 2004
I have noticed that the number of new threads seems to be fairly constant (with less on US weekends) and suspect that, once the number of threads on a given day reaches x, people hold off for fear of their thread being overlooked.
Interesting stuff.
posted by dg at 3:54 PM on July 11, 2004
During that time we had 9/11 happen, tons of new users, and then over a year and a half of no new users, yet the # of comments stays steady.
Wouldn't this be a strong argument for opening up new user registration (the negative effects of new user training periods notwithstanding)?
posted by rushmc at 6:15 PM on July 11, 2004
Wouldn't this be a strong argument for opening up new user registration (the negative effects of new user training periods notwithstanding)?
posted by rushmc at 6:15 PM on July 11, 2004
I agree with dg and LairBob: the front page itself is limited by self-policing, and there's likely a usability cutoff in the forums.
As mentioned above, the interesting statistics to see here would be a) the distribution of comments per post, and b) the growth of FPPs per day since day 1, preferrably normalized by the day-of-week average (or just graph the number of FPPs per week).
I am willing to bet that in the first case, there does exist a cutoff in the number of comments per post (i.e. the distribution is not normal, but has a "cliff" somewhere), just as people have said here, and IMHO, that's probably a) a usability issue with the forum software, and b) a good thing. The fact that the threads are flat, without ratings or other such non-sense, increases the commenting threshold: people think twice about commenting and they won't comment when a thread has become long and tedious. That's technically data-loss (the loss of potential comments that might have been great) but practically it increases the average comment quality. Changing this would turn MeFi into Kuro5hin (not a good thing IMHO).
As far as the number of FPPs per week goes, I am guessing it has asymptotically converged and that's probably unavoidable: there is so much information that the average MeFi user can "consume" per day and doing something about it (like, say, introducing categories to the blue) would lower FPP quality.
In either case, if my guess is right, those are signs of health: users still produce information (FPPs) and they are still willing to participate at the same rate (comments) 5-something years into MeFi's existence which is impressive as online communities go. What we can't see from these stats of course is quality, unless Matt has some pageview data he'd like to share...
posted by costas at 6:42 PM on July 11, 2004
As mentioned above, the interesting statistics to see here would be a) the distribution of comments per post, and b) the growth of FPPs per day since day 1, preferrably normalized by the day-of-week average (or just graph the number of FPPs per week).
I am willing to bet that in the first case, there does exist a cutoff in the number of comments per post (i.e. the distribution is not normal, but has a "cliff" somewhere), just as people have said here, and IMHO, that's probably a) a usability issue with the forum software, and b) a good thing. The fact that the threads are flat, without ratings or other such non-sense, increases the commenting threshold: people think twice about commenting and they won't comment when a thread has become long and tedious. That's technically data-loss (the loss of potential comments that might have been great) but practically it increases the average comment quality. Changing this would turn MeFi into Kuro5hin (not a good thing IMHO).
As far as the number of FPPs per week goes, I am guessing it has asymptotically converged and that's probably unavoidable: there is so much information that the average MeFi user can "consume" per day and doing something about it (like, say, introducing categories to the blue) would lower FPP quality.
In either case, if my guess is right, those are signs of health: users still produce information (FPPs) and they are still willing to participate at the same rate (comments) 5-something years into MeFi's existence which is impressive as online communities go. What we can't see from these stats of course is quality, unless Matt has some pageview data he'd like to share...
posted by costas at 6:42 PM on July 11, 2004
mathowie - is there some information anywhere on the story of the beginning of Mefi? I never realized it until this thread, but Mefi started on my 18th birthday. What else happened on my 18th birthday? I turned 18!
Spooky
But seriously, what is the story of Mefi?
posted by crazy finger at 8:43 PM on July 11, 2004
Spooky
But seriously, what is the story of Mefi?
posted by crazy finger at 8:43 PM on July 11, 2004
well, you see, mefi's mommy and daddy loved each other very much...
posted by quonsar at 10:00 AM on July 12, 2004
posted by quonsar at 10:00 AM on July 12, 2004
(200K comments/year for X users in 2001, and now 200K comments/year for X+Y users in 2004.)
Thanks for that. My brain must have fallen out when I posted my comment. I got that the graph was total comments over time, but somehow reinterpreted it to think that commenting activity had gone up ;-)
posted by wackybrit at 1:50 PM on July 12, 2004
Thanks for that. My brain must have fallen out when I posted my comment. I got that the graph was total comments over time, but somehow reinterpreted it to think that commenting activity had gone up ;-)
posted by wackybrit at 1:50 PM on July 12, 2004
You are not logged in, either login or create an account to post comments
Here's the data:
14-Jul-99 0 (start of mefi)
26-Jun-01 100000
5-Jan-02 200000
3-Jul-02 300000
8-Dec-02 400000
6-Jun-03 500000
12-Dec-03 600000
11-Jul-04 700000
Here's what the first graph of that data looks like:
What is interesting here is that after two years to hit the first 100k, it seems like comments hit all the other 100k marks steadily, at around every six months or so. During that time we had 9/11 happen, tons of new users, and then over a year and a half of no new users, yet the # of comments stays steady. Do you notice how straight the line is after the first 100k? I dropped the first point and fit a line through the rest. Check this out:
The r2 value is almost perfect. That's kind of freaky, maybe we're hitting up against an information overload limit that no amount of new users or events can influence? Any ideas what could be holding us so steady over such changing times?
posted by mathowie (staff) at 9:14 AM on July 11, 2004