Sorting Search Results by Number of Favorites 2: Electric Boogaloo

April 18, 2011 6:42 PM

Pony Request Redux: Adding an option to sort search results by favorites.

Since there are over 600 posts linking to video.google.com, making this megapost required getting in touch with FishBike and politely requesting he employ his stats fu to generate a list of the most popular FPPs featuring Google Video content. This worked out alright -- he managed to scrape the relevant results and sorted them using some custom scripts, ending up with a list of some really stellar stuff -- but it shouldn't be that hard to do.

All this reminded me of a pony request I made nearly two years ago: that we add the option to sort search results by number of favorites. Almost everybody in the thread at the time liked the idea (even some initial critics warmed to it after awhile), and pb said it was technically possible. But I asked it at a pretty bad time -- everybody was gearing up for the big ten-year anniversary celebration, and the request just got lost in the shuffle. I messaged the mods back and forth a bit after the fact for follow-up, but it never really went anywhere. Could we revisit the idea now?

To reiterate my original case for the feature:

- this would be a sorting option, alongside sorting by relevance and by date, but it would not be the default setting.

- it avoids the pitfalls that accompany using favorites as a measure of value in other ways (such as sorting comments in a thread) since the number of favorites would only affect the order of search results -- a unique collection of disparate, mostly archived content being viewed by one person looking for something specific -- not an ongoing discussion in one place viewable to everybody. In other words, there wouldn't be any more incentive to bait for favorites than there already is, since they would only boost the visibility of that comment/post for people searching for the relevant keywords (and then only compared to other content using those keywords). Cracks about Palin or lazy pop culture references would only impact people searching for that content.

- while this would privilege more recent content in search above stuff written before favorites were implemented or widely used, it doesn't discriminate in this regard any more so than the existing sort options -- sorting by date buries older content by default, and sorting by relevance returns what feels like a pretty random assortment of content using basic keyword matching.

- it would make finding interesting content easier. I think most could agree that interesting content on Metafilter tends to attract more favorites -- they're used to generate the "Most Popular" queues, and sidebarred items routinely gather dozens or hundreds of favorites. Adding a favorites sort would be like being able to search the complete archive of these resources all at once.

- it would also make it easy to quickly find items that you can only remember very general keywords for, but which you know had a lot of favorites. For example, Astro Zombie's celebrated Pixar story, perhaps the most popular comment on the site, is on page six of a relevancy search for "pixar" (and even further down using the default "by date" sort), but in a favorites search it would probably be the very first result. I'd love to see what other gems random search terms like that pull up.

- it wouldn't have to be an issue for people who dislike favorites and prefer turning them off -- if they have them toggled off in their profile, then the sort option could just be hidden.


I think it would be good to give this idea a test run using a temporary search page that returns sorted results, just to see how it handles different people's queries and how those results compare to the two standard search types.

This pony's been lost in the wastelands quite awhile... what do you think?

posted by Rhaomi to Feature Requests at 6:42 PM (23 comments total)

I would ride him, and brush him, and feed him apples every day!
posted by killdevil at 6:48 PM on April 18, 2011 [2 favorites]

Pony Request Redux: Adding an option to sort search results by favorites.

You're gonna need to explain why you think this is a good idea.
posted by gman at 7:16 PM on April 18, 2011 [1 favorite]

I think search needs to be fleshed out in any number of ways. How long has 'robust search' been bandied about around here?

Wish I could do more than cheer on; it would be an interesting project.
posted by carsonb at 7:17 PM on April 18, 2011

While favorites are not and can never be a measure of value, it seems to me that they can sometimes in certain circumstances be a measure of notability, which is a useful value in a search function. Rhaomi's case seems sound,

I LIKE THIS PONY!
posted by Blasdelb at 7:25 PM on April 18, 2011 [1 favorite]

Put me down as thinking this is a bad idea. I feel as though favorites have already led to a lot more echo-chamber pile-ons in the past year or two; this isn't to say we're better or worse as a whole across Metafilter, but it's palpable to me, even with favorites turned off, the way the whole system just encourages like-minded people to say more and different-minded people to say less. I've said this before, but I think that's a bad thing. I feel the drawbacks of it even though I have favorites turned off. And I think I would feel it even more if we integrated favorites further into the way people use the site.
posted by koeselitz at 8:36 PM on April 18, 2011

Who uses in-site searching anyway?

Google: site:www.metafilter.com
posted by jpdoane at 8:40 PM on April 18, 2011

That's too bad, koeselitz, especially since you thought it was such a great idea last time around. But like I said, since the sorting will only apply to unique, one-time collections of random comments instead of a central public leaderboard or an ongoing discussion, I don't think any negative effects will be noticeable, especially if it's not the default search option. And the improvements it would bring to the effectiveness of search would definitely outweigh any such effects.

Really, it feels like a waste to have such a strong, pervasive signal of notability throughout the site and then not factor that into search in any way.

jpdoane: "Who uses in-site searching anyway?

Google: site:www.metafilter.com"

Google is powerful, but searches an entire post plus the comments at once; it also doesn't index every single page. The site search is good for when you're trying to find a specific post/comment, or something that post/comment links to (like FishBike and I did for the Google Video posts) -- I don't think Google search inside links like that.
posted by Rhaomi at 8:53 PM on April 18, 2011 [2 favorites]

Posts? Or do you mean posts and comments? Sorting posts by favourites would be... Not horrendous, but pretty unwelcome.

For example.. You couldn't scan through 600 search results for that post? I've only quickly glanced at the post, but assuming you put some editorial effort into that list, I can't imagine scanning through 600 search results would have added significantly to the time required. If you didn't put any editorial effort, because you thought the favourites system could do that for you, well...

Doing it for comments would be horrendous.
posted by Chuckles at 9:41 PM on April 18, 2011

For the record, posts above refers to Front Page Posts and Questions and other topic like things. Comments above refers to comments and answers and other response like things.
I'm sure that is clear enough, but on the other hand I kind of felt like my comment lacked... editorial effort :P
posted by Chuckles at 9:44 PM on April 18, 2011

I like this idea.
I'd also Like this idea if I was on FB, which may explain why I love Favorites.
posted by Lovecraft In Brooklyn at 9:49 PM on April 18, 2011

Doing it for comments would be horrendous.

What? Why? You just assert that, but I don't understand your argument at all. What is the harm? This is only for search results, not sorting whole threads or anything. Think about searching for recipes - it wouldn't be helpful to sort by favorites to find the community's all-time favorite pasta recipe, for example?

If you didn't put any editorial effort, because you thought the favourites system could do that for you, well...

This is really unnecessarily snarky, especially given the obvious effort Rhaomi put into that post. Even if it is currently possible to trawl through 600 posts to find the one you were after, this doesn't really scale up very well as the community (and the archive) grows.
posted by dialetheia at 10:09 PM on April 18, 2011 [1 favorite]

Yeah, posts and comments. I don't see what would be "horrendous" about it.

I mean, take the set of every post/comment that's been sidebarred, and every post/comment that's gotten on to the Popular Favorites page for each subsite. Adding a favorites sort would essentially be like searching these datasets, which hardly seems like a bunch of garbage, but rather some of the best stuff on the site. If it weren't, why would we have these pages in the first place?

Look through the all-time top AskMe questions. Browse this month's most popular posts. Give the Mefi Music charts a listen. Even Projects has monthly toplists. That's all uniformly great content, identified solely through favorites (or votes, for Projects), and these lists are present on every major subsite. Why the revulsion for using the metric that creates these lists as an *option* in search? It wouldn't be flawless -- lots of good content doesn't get a lot of favorites, and plenty of one-off jokes do. But I'm confident they'd return much more interesting results than the relevancy option we have now, which I've never found very useful. This is why I'd like to see a temporary search function, to actually try this idea out and see what kind of results it can generate.

What I really don't see is what's so bad about sorting results this way (optionally!) that people who don't even have favorites turned on in the first place would like to deny that option to folks who would find it quite useful. It provides an immediate, tangible good to those who'd like it, and not much more than a vague, nebulous downside for opponents, one that's already reinforced much more by the aforementioned "top favorites" list for each subsite, whose leaderboard-style sorting encourages competition far more than sorting unique, dynamically-generated search results that only one person sees.

(And yes, getting that sorted list cut down on the time to make the post considerably. It would have taken me days to go through ~660 posts, determine which ones were good to include, test the links to make sure they still worked, and copy over the links + description. Thanks to FishBike, I was able to start from the list of "most popular/bookmarked/liked/etc. Google Video content" and cull from there. It still took a couple of hours, but it was a fraction of the time it would have taken to go through everything manually.)
posted by Rhaomi at 10:12 PM on April 18, 2011 [1 favorite]

To put it one more way, look at one's profile history (which is akin to doing a site search for all content posted by user X). If you look at mine, the "view all activity" page returns 89 posts and 3,274 comments across all sites. That's a buttload of content, and I'm not even one of the most prolific posters here. If you want to browse the best of this, going chronologically isn't too useful. But go to my top favorites, and bam -- you get what I consider to be my top contributions to the site. Two of the top three won best post contests. The top comment made the sidebar. #8, a concise summary of Lost, was reblogged all over the place.

Again, it's not perfect -- #11 is a one-liner about Juan Williams. #13 is a time zone joke. But those relative duds are far outnumbered by Dune Encyclopedias and Planet Earths and A List Of The 25 Funniest Mefi Comments I've Ever Seen. It's a hell of a lot more interesting than looking at them by date, and I'm sure it's better than whatever the current relevancy search would turn up. And I think this is true for most everybody on the site -- one's most popular contributions are generally the best ones. It's not a 1:1 relationship, but the correlation is strong enough to be interesting, and that's a good quality to have in search.
posted by Rhaomi at 10:24 PM on April 18, 2011

I would like this pony for posts and comments., and don't really see any downside despite vague noises from people who think the whole favorites system is bad for the site.

If you didn't put any editorial effort, because you thought the favourites system could do that for you, well...

ha ha ha what? I feel like Rhaomi's trademark here is the extensive research he puts into every post he's made, as a brief scan of his (uniformly excellent) posting history would have told you.
posted by lalex at 10:56 PM on April 18, 2011 [1 favorite]

It's a good idea, I can see why some people might have misgivings about it, but I think we are a strong enough community to not lurch into FB territory.
This thread is worth it for me for the link to Astro Zombie's comment which I missed first time around.
posted by arcticseal at 2:01 AM on April 19, 2011

You're gonna need to explain why you think this is a good idea.

That doesn't seem entirely fair, I think she explained at length in the post why?
posted by Meatbomb at 2:22 AM on April 19, 2011

...I think gman's comment was about as serious as I am female.

(not that there's anything wrong with that :P)
posted by Rhaomi at 3:27 AM on April 19, 2011

I would like to see this idea implemented.

With the facilities provided by this site itself, plus the availability of the Infodump and various tools to analyze that, we already have many different ways to search for things and see the results with the highest favorite counts. Every time I do that, I find I get a list of mostly really good stuff.

While I think the whole issue of whether or not the favorites mechanism encourages undesirable behavior is one where people can reasonably disagree, I'm just not seeing that problem reflected in the results we get already, with the possible exception of Metatalk comments, where a few moderator announcements of bannings have been very highly favorited.

I seem to remember mathowie saying something like the purpose of the favorites system is to help people find good stuff. I think that's a great summary of what the mechanism does in a number of different ways, and I think it works well for that. Being able to sort search results by favorite count would just make it even more useful.
posted by FishBike at 5:32 AM on April 19, 2011

I think where we are on this on Team Mod is that search-by-favorites isn't something we're so fundamentally interested in doing that we're going to go out of our way to make it happen prior to the point where we're finally able to tackle a real overhaul of the search interface.

I can see both how it would be useful for specific sorts of searching and why people would have concerns about it being used as a metric or about the effect that endorsement might have culturally; maybe it's just a foggy morning for me but I don't have any super strong feelings about it either way, and the bad news for proponents is that if we're all in "eh..." territory that's not a good sign for it jumping to the top of the priority list.
posted by cortex (staff) at 7:24 AM on April 19, 2011

That's discouraging, I've got to say -- I think the arguments for it are pretty strong as far as improving the utility of search goes.

Could we at least give it a test run? Not integrate it into the official search for everybody, but set up a special purpose page temporarily for the purposes of this thread, like has been done for some test features in the past, in order to see how sorted results compare to a regular search. I think the case for it would be stronger if it's actually proven to return higher-quality results for arbitrary queries as far as people here can judge. Heck, it might even work better as a more obscure MarkovFilter/Tag Cloud/Social Explorer kind of specialty page rather than included as an option in regular search -- useful and interesting to those who want it, but minimizing the impact on perceptions by not attaching it to every single search.

But really, I'd just love to powerbrowse like this for once -- what are the most popular flash games ever posted to the site? The best YouTube link posted as a comment? The top William Carlos Williams parody? The most bookmarked/recommended Amazon link or TVTropes page or PDF or recipe? Etc. The Google Video post wasn't the first time I've wanted to tap into the crowdsourcing utility of favorites like that. Combined with being able to search by subsite, tag, and link URL, I think this has a strong potential to return very useful and interesting results for a wide range of cases, like it already does for user profiles and the various toplists. It's already acknowledged as useful for this purposes and clearly turns up consistently good content; it feels like a waste to not at least give it a shot with search and see how it performs.
posted by Rhaomi at 8:22 AM on April 19, 2011

That's discouraging, I've got to say -- I think the arguments for it are pretty strong as far as improving the utility of search goes.

Well, from where I look at it there's strong arguments for a number of kinds of search improvement—bucketing by date range, for example, would be huge improvement—but none of that is free and of the things we'd like to overhaul for a more powerful search interface sort-by-favorites isn't topping the list.

So it's not really an "a waste to not at least give it a shot" situation so much as a "why is this the thing that needs to be at the forefront of our attention" thing. I appreciate your arguments for it, and like I said I don't really have a big problem with the idea and don't know that the rest of the team does either, but it's not in This Is What We're Doing Now territory either.
posted by cortex (staff) at 8:37 AM on April 19, 2011

It just felt like the easiest and most logical extension of the feature, is all, along with providing the most bang for one's programming buck. Date ranging would be pretty complicated UI-wise, for instance, and doesn't offer much additional utility over skipping through a by-date search to the time period you want. But this pony, IMHO, greatly improves the relevance of search over what's available now, in a way you can't replicate manually at the moment. Most of the improvements to search I can think of can be bootstrapped fairly easily one way or another -- I've wanted to be able to sort user history search by subsite in the past, for example, which I can get around by just Ctrl+F'ing each page for the relevant subsite name -- but you can't use favorites without scraping the raw data from the Infodump yourself. You can't even see that information in search without clicking through to each result.

So it doesn't really make one aspect easier in competition with other incremental improvements so much as it adds a whole new feature that's been conspicuously absent up to now. I really can't think of a search improvement suggested in the last few years that would improve the experience of search as much as I think this will (apart from maybe supporting search operators like NOT and exact phrases, which Google can handle imperfectly.)
posted by Rhaomi at 9:18 AM on April 19, 2011

(And by "easiest," I just mean that it's a simple number already attached to each item for ready sorting, as opposed to something more involved like a smarter search algorithm or real-time suggestions. Not saying it's a breeze to implement though, knowing next to nothing about programming.)
posted by Rhaomi at 9:35 AM on April 19, 2011