How does Aardvark compare to Ask Metafilter? February 4, 2010 7:27 AM Subscribe
How does Aardvark compare to Ask Metafilter?
Aardvark recently published a paper Anatomy of a Large-Scale Social Search Engine, in which they make certain claims about their numbers, such as:
-------------------------
87.7% of questions sent to Aardvark got answered (very high answer rate!)
75.0% of users who asked Aardvark a question also answered a question for someone else (very high participation rate!)
70.4% of answer feedback had a rating of ‘good’ as opposed to ‘ok’ or ‘bad’ (high quality!)
-------------------------
Aardvark is a pretty nice service, and well thought out, but it seems like all of the "revolutionary" results are also provided by AskMe, albeit possibly in a less obviously "social" way (and certainly with fewer entry points). Particularly, it seems like the results on AskMe aren't really any substantially less good for not being targeted at an explicit network of friends, or really being a "search engine".
So I'm curious - how does AskMe compare to Aardvark, both on those numbers (which seem in my limited survey to be much higher for AskMe), but also on whatever other axis you care to discuss. I'm interested in your thoughts.
Aardvark recently published a paper Anatomy of a Large-Scale Social Search Engine, in which they make certain claims about their numbers, such as:
-------------------------
87.7% of questions sent to Aardvark got answered (very high answer rate!)
75.0% of users who asked Aardvark a question also answered a question for someone else (very high participation rate!)
70.4% of answer feedback had a rating of ‘good’ as opposed to ‘ok’ or ‘bad’ (high quality!)
-------------------------
Aardvark is a pretty nice service, and well thought out, but it seems like all of the "revolutionary" results are also provided by AskMe, albeit possibly in a less obviously "social" way (and certainly with fewer entry points). Particularly, it seems like the results on AskMe aren't really any substantially less good for not being targeted at an explicit network of friends, or really being a "search engine".
So I'm curious - how does AskMe compare to Aardvark, both on those numbers (which seem in my limited survey to be much higher for AskMe), but also on whatever other axis you care to discuss. I'm interested in your thoughts.
The aardvark asked for an aardvark.
posted by yeti at 7:37 AM on February 4, 2010 [1 favorite]
posted by yeti at 7:37 AM on February 4, 2010 [1 favorite]
My feeling about Aardvark is it's really good for quick reference sorts of questions where there's one answer [or a set of answers "BBQ joints in Boston" or something] and you'd like it relatively quickly. I was really turned off by them initially because when I went to SXSW the year they were in beta, they seemed to be doing that classic SXSW thing where they'd show up at other people's panels to ask questions that were thinly veiled promotional pieces for their own site/software.
The big difference that I see between them and us [and I wouldn't be making the comparison unless someone specifcally asked] is that we don't publish papers about how awesome we are. Not that there's anything wrong with that, per se, but I sort of can't shake the feeling I got at SXSW that Aarvark is more interested in being seen as useful than being useful. Like do you really need to tell me that 87.7% is a "high answer rate"? The whole thing reads strangely.
That said I've seen other people saying that they find it useful -- usually highly connected folks for whom adding this sort of thing to their IM list is simple and sensible -- so this may just be me being very comfy with what I'm already using/doing. And, as an aside, I find that they've oversimplified the library/village distinction as if all librarians are doing is searching Google better than you. While there's a sense in which that's true, knowledge of reference materials and what we call the "reference interview" is a pretty important part of why people still go to the library and don't just ask their friends all their questions.
posted by jessamyn (staff) at 7:38 AM on February 4, 2010 [7 favorites]
The big difference that I see between them and us [and I wouldn't be making the comparison unless someone specifcally asked] is that we don't publish papers about how awesome we are. Not that there's anything wrong with that, per se, but I sort of can't shake the feeling I got at SXSW that Aarvark is more interested in being seen as useful than being useful. Like do you really need to tell me that 87.7% is a "high answer rate"? The whole thing reads strangely.
That said I've seen other people saying that they find it useful -- usually highly connected folks for whom adding this sort of thing to their IM list is simple and sensible -- so this may just be me being very comfy with what I'm already using/doing. And, as an aside, I find that they've oversimplified the library/village distinction as if all librarians are doing is searching Google better than you. While there's a sense in which that's true, knowledge of reference materials and what we call the "reference interview" is a pretty important part of why people still go to the library and don't just ask their friends all their questions.
posted by jessamyn (staff) at 7:38 AM on February 4, 2010 [7 favorites]
To be honest, I probably only participate with Aardvark because they've made it ridiculously easy for me to do so via IM in a way that's fairly unobtrusive and I'm logged in anyway. That, I think, is the real benefit they've added so far.
posted by Caviar at 7:44 AM on February 4, 2010
posted by Caviar at 7:44 AM on February 4, 2010
If one of the other people who have worked with the Infodump don't beat me to it, I'll see if I can get those numbers out of the data sometime later today. At least the first two numbers (percentage of questions getting an answer and people asking questions who also participate) ought to be significantly higher.
My WAG is that the unanswered-question rate is under 1%, but we'll see. I also think that the number of people asking questions who then immediately disappear is very low, far less than Aardvark's 25%.
The third number doesn't have a direct corollary on AskMeFi; on AskMe, questions get marked as having a 'best answer' (or multiple best answers) or they don't. There's no "ok" or "bad" options. So we'd have to think about what the best comparison is there.
posted by Kadin2048 at 7:55 AM on February 4, 2010
My WAG is that the unanswered-question rate is under 1%, but we'll see. I also think that the number of people asking questions who then immediately disappear is very low, far less than Aardvark's 25%.
The third number doesn't have a direct corollary on AskMeFi; on AskMe, questions get marked as having a 'best answer' (or multiple best answers) or they don't. There's no "ok" or "bad" options. So we'd have to think about what the best comparison is there.
posted by Kadin2048 at 7:55 AM on February 4, 2010
My WAG is that the unanswered-question rate is under 1%, but we'll see.
One thing worth noting is that although nearly every AskMe question gets comments of some sort, they are not always answers in the sense that they actually answer the question. For example this sketch identification question has a lot of comments, but the end result was that the asker didn't really find out the answer.
posted by burnmp3s at 8:03 AM on February 4, 2010
One thing worth noting is that although nearly every AskMe question gets comments of some sort, they are not always answers in the sense that they actually answer the question. For example this sketch identification question has a lot of comments, but the end result was that the asker didn't really find out the answer.
posted by burnmp3s at 8:03 AM on February 4, 2010
Yeah, that's true; the "answer" rate for AskMe is probably best thought of as somewhere between the "questions marked as having a best answer" percentage and "questions getting some response at all." If you look only at best answers AskMe might seem low, if you look at any response at all, it'd look high. But without capturing a lot more data (AskMe Asker Satisfaction Survey?) I don't think you can really tell better than that.
posted by Kadin2048 at 8:06 AM on February 4, 2010
posted by Kadin2048 at 8:06 AM on February 4, 2010
One could look at the percentage of (non-anonymous) questions that have either a resolved tag or a best answer marked. One would want to limit the analysis to questions asked after the resolved tag was introduced. You'd still have some false negatives, though.
posted by jedicus at 8:15 AM on February 4, 2010
posted by jedicus at 8:15 AM on February 4, 2010
Previously on the blue, and in particular heresiarch's comment.
posted by serathen at 8:19 AM on February 4, 2010
posted by serathen at 8:19 AM on February 4, 2010
The ways you might measure quality of feedback are fraught with problems. The 'unresolved' tag is rarely used, but some of the questions still have best answers marked. The 'stumped' tag, which I added to a question of mine that got zero answers is apparently used by other people because they are stumped by the problem they are asking about, not because AskMe is stumped. As a result, there is a subset that also have a 'resolved' tag. I'm not sure how you would subset the questions that were poorly answered using our current data.
posted by OmieWise at 8:22 AM on February 4, 2010 [1 favorite]
posted by OmieWise at 8:22 AM on February 4, 2010 [1 favorite]
If you killed an aardvark, how would you dispose of the body without getting caught?
posted by Elmore at 8:31 AM on February 4, 2010
posted by Elmore at 8:31 AM on February 4, 2010
75.0% of users who asked Aardvark a question also answered a question for someone else (very high participation rate!)
I don't understand how you got that interpretation from that statistic. If I'm asking questions for someone else, doesn't that mean the "participation" is low? That other person isn't participating (i.e. posting their own answers to questions, etc).
70.4% of answer feedback had a rating of ‘good’ as opposed to ‘ok’ or ‘bad’ (high quality!)
"good" or "ok" relative to what? If I asked a question about tying my shooes on Yahoo Answers and didn't end up dead, I'd consider that "good". How do you compare that across sites?
posted by DU at 8:55 AM on February 4, 2010
I don't understand how you got that interpretation from that statistic. If I'm asking questions for someone else, doesn't that mean the "participation" is low? That other person isn't participating (i.e. posting their own answers to questions, etc).
70.4% of answer feedback had a rating of ‘good’ as opposed to ‘ok’ or ‘bad’ (high quality!)
"good" or "ok" relative to what? If I asked a question about tying my shooes on Yahoo Answers and didn't end up dead, I'd consider that "good". How do you compare that across sites?
posted by DU at 8:55 AM on February 4, 2010
I think you're misparsing that first one, DU. The assertion is that 75% of folks who asked their own question also provide at least one answer to some other asker's question. That they give as well as receive, in other words.
posted by cortex (staff) at 9:06 AM on February 4, 2010
posted by cortex (staff) at 9:06 AM on February 4, 2010
All the questions on Ask Yahoo are answered, but most are incorrect, offensive, or just plain stupid.
posted by blue_beetle at 9:31 AM on February 4, 2010
posted by blue_beetle at 9:31 AM on February 4, 2010
Oh, I see. *Answered* a question for someone else. Right, complete misread on my part.
posted by DU at 9:59 AM on February 4, 2010
posted by DU at 9:59 AM on February 4, 2010
What if there were some meta-Answer engine that would post your question to every appropriate forum on the internet? And then it would use a complex algorithm to get rid of stupid answers?
It'd pollute a bunch of forums with inappropriate questions, but in the quest for answers, there are no rules. If someone doesn't pick up the phone, you have every right to lie your way into their office and mess with their phone system to see what's up.
posted by mccarty.tim at 10:00 AM on February 4, 2010
It'd pollute a bunch of forums with inappropriate questions, but in the quest for answers, there are no rules. If someone doesn't pick up the phone, you have every right to lie your way into their office and mess with their phone system to see what's up.
posted by mccarty.tim at 10:00 AM on February 4, 2010
Ha, oh god, that blog post is hilarious. I like Aardvark, but it is in no way a replacement for google so I'm really weirded out that they would seem to imply that.
posted by The Devil Tesla at 10:20 AM on February 4, 2010
posted by The Devil Tesla at 10:20 AM on February 4, 2010
I have an intense urge to smite Aardvark, just because a friend of mine has his account set up to post to his Facebook and Twitter accounts and mail to a list whenever he answers someone.
And he answers a LOT of people.
This is why he's blocked by people he knows.
posted by mephron at 10:55 AM on February 4, 2010 [1 favorite]
And he answers a LOT of people.
This is why he's blocked by people he knows.
posted by mephron at 10:55 AM on February 4, 2010 [1 favorite]
Big deal, I registered Aardball.com to be a cross between Aarvark and Hardball... never quite figured that one out. Anybody wanna buy a domain?
posted by oneswellfoop at 11:47 AM on February 4, 2010
posted by oneswellfoop at 11:47 AM on February 4, 2010
I use Aardvark for frequent dumb questions, Ask Metafilter for relatively important questions. I'm not a huge Aardvark fangirl, but it's a nice thing to have around.
Mostly I have zillions of identification questions about things I see around town — weird structures, etc. — so I often send Aardvark a photo and a question about it. I sometimes get lucky with an expert who has the precise answer to my question, but most of the time I get non-answers from nice people who are just as clueless as me. I also ask Aardvark for help coming up with the right word, rephrasing a sentence, etc. — something where I just want another pair of eyes.
There's a very low "cost" to asking questions on Aardvark, so it doesn't hurt to send it lots of short questions. I'm not limited to one question a week, and I don't feel like I'm wasting anyone's time (or pushing down good questions) if I ask something unimportant.
I go to Ask Metafilter when I have a question that I really want answered well, and I spend some time trying to write a clear and full question. I get thoughtful, useful answers every time.
posted by dreamyshade at 1:18 PM on February 4, 2010
Mostly I have zillions of identification questions about things I see around town — weird structures, etc. — so I often send Aardvark a photo and a question about it. I sometimes get lucky with an expert who has the precise answer to my question, but most of the time I get non-answers from nice people who are just as clueless as me. I also ask Aardvark for help coming up with the right word, rephrasing a sentence, etc. — something where I just want another pair of eyes.
There's a very low "cost" to asking questions on Aardvark, so it doesn't hurt to send it lots of short questions. I'm not limited to one question a week, and I don't feel like I'm wasting anyone's time (or pushing down good questions) if I ask something unimportant.
I go to Ask Metafilter when I have a question that I really want answered well, and I spend some time trying to write a clear and full question. I get thoughtful, useful answers every time.
posted by dreamyshade at 1:18 PM on February 4, 2010
I favorited Meph just because he and I know the same guy, and holy shit is that constant Aardvark spam annoying.
posted by fairytale of los angeles at 2:25 PM on February 4, 2010 [1 favorite]
posted by fairytale of los angeles at 2:25 PM on February 4, 2010 [1 favorite]
I was really turned off by them initially because when I went to SXSW the year they were in beta, they seemed to be doing that classic SXSW thing where they'd show up at other people's panels to ask questions that were thinly veiled promotional pieces for their own site/software.
That perfectly describes exactly the sort of spidey-sense tingle I get from this MeTa post.
Of course, though, I am not, in fact, a spider-man, so that tingle probably just means the shampoo is working.
posted by Sys Rq at 2:29 PM on February 4, 2010
That perfectly describes exactly the sort of spidey-sense tingle I get from this MeTa post.
Of course, though, I am not, in fact, a spider-man, so that tingle probably just means the shampoo is working.
posted by Sys Rq at 2:29 PM on February 4, 2010
That perfectly describes exactly the sort of spidey-sense tingle I get from this MeTa post.
You and me both, actually. But I did a little typa typa and didn't see anything obvious, so I figured I'd shut up and see what happened. Caviar, just to get this out of the way... your employer doesn't work for Aardvark, do they?
posted by jessamyn (staff) at 2:36 PM on February 4, 2010
You and me both, actually. But I did a little typa typa and didn't see anything obvious, so I figured I'd shut up and see what happened. Caviar, just to get this out of the way... your employer doesn't work for Aardvark, do they?
posted by jessamyn (staff) at 2:36 PM on February 4, 2010
oneswellfoop: “Big deal, I registered Aardball.com to be a cross between Aarvark and Hardball... never quite figured that one out. Anybody wanna buy a domain?”
What in god's name were you thinking? Aardball? Hardvark.com is obviously superior, isn't it?
posted by koeselitz at 4:45 PM on February 4, 2010
What in god's name were you thinking? Aardball? Hardvark.com is obviously superior, isn't it?
posted by koeselitz at 4:45 PM on February 4, 2010
Harkvark.com is the porn parody.
posted by shakespeherian at 7:41 AM on February 5, 2010
posted by shakespeherian at 7:41 AM on February 5, 2010
Caviar, just to get this out of the way... your employer doesn't work for Aardvark, do they?
No answer? Hmm.
posted by ThePinkSuperhero at 5:45 PM on February 5, 2010
No answer? Hmm.
posted by ThePinkSuperhero at 5:45 PM on February 5, 2010
I both ask and answer questions on Aardvark, but to be honest, I'd say the quality's fairly low a lot of the time. I occasionally get a good answer, but most of the time it's guess work and I rarely get clarifying questions. I certainly wouldn't ask anything long or complicated. The quality and frequency of questions have both decreased over time in my experience.
posted by eleanna at 6:35 PM on February 5, 2010
posted by eleanna at 6:35 PM on February 5, 2010
Caviar, just to get this out of the way... your employer doesn't work for Aardvark, do they?
Firmly for the record I have no affiliation with Aardvark whatsoever (and my employer, generally, is me), with the sole exception being that I have an account. I'm a longtime Ask Metafilter fan, I've been using Aardvark recently, and this just piqued my interest.
posted by Caviar at 6:03 PM on February 6, 2010
Firmly for the record I have no affiliation with Aardvark whatsoever (and my employer, generally, is me), with the sole exception being that I have an account. I'm a longtime Ask Metafilter fan, I've been using Aardvark recently, and this just piqued my interest.
posted by Caviar at 6:03 PM on February 6, 2010
a friend of mine has his account set up to post to his Facebook and Twitter accounts and mail to a list whenever he [X]
It's one of my general rules that unless the facebook or twitter account or mailing list in question exists solely for the purpose of monitoring said events, this is pretty much never a good idea, regardless of what X is.
posted by Caviar at 1:34 PM on February 7, 2010
It's one of my general rules that unless the facebook or twitter account or mailing list in question exists solely for the purpose of monitoring said events, this is pretty much never a good idea, regardless of what X is.
posted by Caviar at 1:34 PM on February 7, 2010
In answer to your original question... well Google hasn't bought us.
posted by jessamyn (staff) at 9:53 AM on February 11, 2010
posted by jessamyn (staff) at 9:53 AM on February 11, 2010
In answer to your original question... well Google hasn't bought us.
Considering your source there is TechCrunch, I wouldn't be surprised if Google hasn't bought Aardvark either.
posted by The Devil Tesla at 11:11 AM on February 11, 2010 [1 favorite]
Considering your source there is TechCrunch, I wouldn't be surprised if Google hasn't bought Aardvark either.
posted by The Devil Tesla at 11:11 AM on February 11, 2010 [1 favorite]
(Well, it's now confirmed, the TechCrunch article used to just have "our source" saying it happened. That makes it a rumor, and TC should have titled that article in some way that suggested that.)
posted by The Devil Tesla at 7:13 AM on February 12, 2010
posted by The Devil Tesla at 7:13 AM on February 12, 2010
I suppose I have to go on record now as saying I don't have any affiliation with Google either. I do have a long list of blog posts criticizing their privacy practices, but I don't think anyone really pays attention to that and I've mostly stopped bothering.
posted by Caviar at 10:19 AM on February 16, 2010
posted by Caviar at 10:19 AM on February 16, 2010
Hopefully it isn't too late to post some MeFi stats for comparison. Looking at stats for questions posted in 2009 only:
31,103 questions were posted in 2009.posted by FishBike at 6:38 PM on February 24, 2010 [1 favorite]
2,543 were anonymous questions.
28,560 were non-anonymous questions.
30,872 questions had at least 1 answer by someone other than the poster. That's 99.26% of all questions that got an answer.
8,446 distinct users posted questions in 2009.
8,082 of those (95.69%) have answered at least one question that was not their own, at some time (not limited to 2009 only).
12,800 non-anonymous questions had at least 1 "best answer" by someone other than the asker. That's 44.82% of non-anonymous questions.
17 anonymous questions also had at least 1 "best answer" marked. This can happen when the question is anonymized later, after a best answer has already been marked by the asker.
The green comes out looking pretty good there, doesn't it?
I wonder how it would look if more detail was available about anonymous questions? Potentially there could be as many as 2,543 unique users who never answer any questions, so that could reduce one of the stats a fair bit. Or it could be 1 person who asked all 2,543 anonymous questions, which is how it's counted in the "percent of askers who also answered" stats right now.
However, given all the reasons why people ask questions anonymously, I'm not sure it's wise to go digging into that any further. Not that it's possible anyway with what's available in the Infodump (in case anyone's worried about that possibility).
The metrics about perceived answer quality are pretty hard to compare, since we don't have a system of directly rating that. I was considering trying to figure out what percentage of answers are marked "best answer" OR favorited by at least one person, as some kind of proxy indication that it is a "good" answer.
But that doesn't seem like a very accurate indication, given what favorites are1 and how they are used.
1: They are quanta of happiness.
posted by FishBike at 7:28 AM on February 25, 2010
I wonder how it would look if more detail was available about anonymous questions? Potentially there could be as many as 2,543 unique users who never answer any questions, so that could reduce one of the stats a fair bit. Or it could be 1 person who asked all 2,543 anonymous questions, which is how it's counted in the "percent of askers who also answered" stats right now.
However, given all the reasons why people ask questions anonymously, I'm not sure it's wise to go digging into that any further. Not that it's possible anyway with what's available in the Infodump (in case anyone's worried about that possibility).
The metrics about perceived answer quality are pretty hard to compare, since we don't have a system of directly rating that. I was considering trying to figure out what percentage of answers are marked "best answer" OR favorited by at least one person, as some kind of proxy indication that it is a "good" answer.
But that doesn't seem like a very accurate indication, given what favorites are1 and how they are used.
1: They are quanta of happiness.
posted by FishBike at 7:28 AM on February 25, 2010
You are not logged in, either login or create an account to post comments
The third line doesn't really make for direct comparison, since we have no explicit granular rating system. It'd be interesting to look at the rate of questions that receive at least one best answer, I suppose, though as the system doesn't try very hard to cause people to mark those it's still only a general analogue. I suppose I could try and crunch some numbers on flagging (negative, neutral, and fantastic) for another sort of aggregate view of some of that, but I don't know if I'll really get around to that right now.
posted by cortex (staff) at 7:32 AM on February 4, 2010