Ponies: a comprehensive study in equine requesting practices (MetaFilter, 2011)

May 1, 2011 3:59 PM

What if there was a section of the site dedicated to research projects related to MetaFilter?

As I'm reading the site and its subsections I'm always asking myself questions and thinking of research methods that could answer them, a few recent examples:

• What does it mean when different people favorite a comment? What are the circumstances that someone clicks that plus sign, and what's going through their head as they do it? An ethnographic study of a variety across a wide spectrum of activity might work towards answering that.
• How do different users, with diverse backgrounds and levels of education, decide to tag posts? Could tagging be improved? A survey could be conducted that asks users to describe their tagging practices, and rate the efficacy of tags in finding information.
• How do controversial threads (the Japan Nuclear Crisis, for example) effect site sign-ups, comment activity, etc. This would involve studying more admin-side data than anything else.

Those are just a few examples.

The problem is that this research is labor-intensive, and probably not worth the investment for an individual, but as a collective it would be a lot easier. The subsection I'm picturing would have proposal submissions, and then a regular selection of questions to tackle. It might even be a way for site members to publish research that's slightly below journal standards, but would still give practice doing research and provide a jumping off point for more academic level work. A veritable university of bean-plating.

I'm curious what other people think, and the mods' opinions on the matter.

posted by codacorolla to Feature Requests at 3:59 PM (13 comments total)

Perfect for the wiki, not something that makes sense to maintain on the main site, in my personal opinion.
posted by jessamyn (staff) at 4:02 PM on May 1, 2011 [1 favorite]

Yeah, agreed; it's not really necessary an Official Metafilter Organ sort of thing I don't think, I'm not sure where on the site itself it would necessarily go.

That said, I like the idea and I would honestly love to see some research-minded (or organization-minded research-curious) people go to town on stuff like this on the wiki. Between the Infodump and random datawankery people get up to around here, and the occasional actual straight up academic research done involving Mefi, making some sort of more well-constructed home for existing and future analysis and such of Mefi would be great.

I could see maybe making a sort of bibliography page of notable research stuff on the stuff.metafilter.com domain—I'm thinking literally just a page that maintains links to papers, presentations, and just interesting in-thread analyses, that sort of thing. But that'd be more of a static document I'd update occasionally, whereas the wiki is something anyone feeling enthusiastic can contribute to or edit, so wiki as staging/organizing ground for this stuff would probably be the best approach.
posted by cortex (staff) at 4:11 PM on May 1, 2011

As far as all that goes, what's on the wiki already is the Infodump details, a similar but much sparser at this point page of Corpus Project details, and a big and already loosely organized collection of links to research stuff on the MetaAnalysis page.

I feel like step one would be looking at that last one and trying to figure out if there's a good way to add further structure to it and/or incorporate its existing contents into a more ambitiously structured view into existing and potential future investigative tomfoolery.
posted by cortex (staff) at 4:14 PM on May 1, 2011

Would you guys consider linking or side barring research projects that were in progress as a way to get participants for study? Also, I'm not sure how the ethics would play out in terms of getting people to give consent to research. It's not like there's an IRB for the Internet.
posted by codacorolla at 4:14 PM on May 1, 2011

I often find it weird enough when the site examines itself. When metafilter is mentioned someplace else and that gets posted here and there are the invariable dissections of what the author got right or wrong (and the disagreements related to this). Or when someone comes along and asks permission to study the site for a school project or dissertation. Or when the site accomplishes some real world thing that people think is neat.

I think a section of this site would be like when you are looking into a mirror with one at your back and you see yourself going on forever and ever smaller.

This is not an endorsement or condemnation of the idea. It's just not something I'd be comfortable reading or contributing to. I'd also wonder at how much of a sitewide participation discouragement such studies could be. I doubt I'd be unable to engage the site in any substantive manner without wondering what someone out there would think about my interaction.

I like seeing a lot of these studies, but I like that they seem to develop organically.

I am not a number! I am? Ignore me then. User 59453.
posted by cjorgensen at 4:16 PM on May 1, 2011

Would you guys consider linking or side barring research projects that were in progress as a way to get participants for study?

We've typically done that when someone was doing a bit of serious academic work, yeah; if it's something where "go answer this questionnaire FOR SCIENCE!" would help get eyes on a project, sidebarring it is usually a workable thing to do.

It'd depend to some extent on the project, of course, and the volume of projects we were looking at. If there were a lot of smallish just-for-a-hoot things going on, that might be a situation where sidebarring wasn't always in the cards.

Also, I'm not sure how the ethics would play out in terms of getting people to give consent to research. It's not like there's an IRB for the Internet.

Well, there's some tricky things there. I think the spirit of the thing is important: if someone wanted to do the sort of research that would normally get the attention of an IRB in an academic environment, we'd want to know what the plan was and would need for people on the site to have a chance to basically talk about and opt into any such stuff.

If there was a question of like surreptitiously monitoring on-site behavior or doing some sort of stimulus-and-response thing where the researcher "plants" behavior on the site, that would be more in pretty questionable territory, IRB or not.

What we've typically seen with research we've been okay with is either polls with an explicit opt-in nature, or examinations of site-activity-as-public-artifact, looking at public data that's already fixed in stone, which is generally pretty unobjectionable if care is taken not to do anything weird with the results to shine a bright spotlight on individuals without their permission.

But in the context of this idea, it wouldn't be a bad idea to at least lay some of those ideas out explicitly.
posted by cortex (staff) at 4:25 PM on May 1, 2011

One thing to consider is that most of us research-y types work within a theory or at least have research agendas that makes 'one offs' like these (fun and interesting) research questions harder to tackle. (A whole new literature to learn, time, justification for getting 'off track')... even though I consider it often! :)
posted by k8t at 4:52 PM on May 1, 2011

Hey, I actually did a study on how useful tags were in finding information, using AskMe to stand in for tagged information in text form. I'm still writing up that part, and was wondering how I would eventually communicate to the community what I found.

k8t brings up a good point regarding situating the type of research you're proposing in academia. If it's for academic publication, there's no incentive to post it to some MeFi site prior to publication in an academic conference or journal. People do smallish research studies for say for a class project, but typically they don't request separate IRB approval for it, using blanket IRB approval for coursework. And in that case you're not supposed to publish the work. So it's not clear what the motivation would be to do "research that's slightly below journal standards, but would still give practice doing research and provide a jumping off point for more academic level work." You don't get academic benefit from doing it, and there might be academic rules against doing it.
posted by research monkey at 5:01 PM on May 1, 2011 [1 favorite]

Official Metafilter Organ

I can't be the only person that thought of Videodrome, right? Now that would make a hell of a research project.
posted by AkzidenzGrotesk at 5:30 PM on May 1, 2011

Official Metafilter Organ

I can't be the only one with that tattoo
posted by found missing at 5:35 PM on May 1, 2011

Self-examination pony
posted by Favorites Pony at 6:29 PM on May 1, 2011

What if there was a section of the site dedicated to all of the times I have made this exact comment right now?
posted by Plutor at 8:42 AM on May 2, 2011

OK, it seemed like a good idea, but probably isn't feasible. Thanks for the feedback.
posted by codacorolla at 11:40 AM on May 2, 2011