Desperately seeking search

December 31, 2005 6:08 AM

Please bring back some kind of built-in search capability.
posted by killdevil to Feature Requests at 6:08 AM (26 comments total)

It's very difficult to locate posts of interest or check for duplicates using only the Yahoo or Google index of Metafilter. So, this is another request for the return of native post and comment search functionality.

Alternatively, there was some musing about offloading structured site search into Google Base -- have there been any developments on that front?
posted by killdevil at 6:09 AM on December 31, 2005

Second.
posted by mediareport at 6:10 AM on December 31, 2005

Yahoo works astonishingly well for the site, I've found. The duplicate URL-checker works as advertised. What 'built-in' search we ever had never worked worth a damn.

Of course a real, granular search page (for the first time ever) would be welcome. I'm not going to hold my breath, though.
posted by stavrosthewonderchicken at 6:27 AM on December 31, 2005

can you explain how it's difficult? i just followed the "search" link, typed "coffee" in a box, clicked on a button, and saw a list of references to coffee on metafilter. how much easier do you want?
posted by andrew cooke at 6:28 AM on December 31, 2005

you mean a partial list of a few references to coffee on metafilter.
posted by quonsar at 7:16 AM on December 31, 2005

hmm. i guess you're right. searching for "andrew cooke" gives 666 links. so i guess it's either incomplete or channeling information from another world.
posted by andrew cooke at 7:19 AM on December 31, 2005

As quonsar says, the trouble is that the list of results is incomplete and unstructured. If I half-remember a particular post about coffee, and that it was posted within a certain range of dates, I can't isolate it from the clutter of all the references to the word coffee that show up in Yahoo's index.

I'd be thrilled to see a posts-and-tags-only internal search with an optional date range selector. Comments aren't as important, truthfully, and I can see how trolling through a million comments with a SQL query might stretch a single-server setup to the breaking point.
posted by killdevil at 7:27 AM on December 31, 2005

Yeah, searching with Yahoo is crap. The old system worked really well.
posted by cillit bang at 7:46 AM on December 31, 2005

666 links. so i guess it's either incomplete or channeling information from another world.

the search of the beast!

*waves crucifix*
posted by quonsar at 7:49 AM on December 31, 2005

It's crap because you can't just search in AskMe questions, you have to search all the answers as well.
posted by bonaldi at 8:40 AM on December 31, 2005

The old system worked really well.

Er, other than the "crashed the whole site regularly" part, you mean? That's why it's not here anymore.
posted by mendel at 9:22 AM on December 31, 2005

The thing I miss most is being able to sort the results by date of posting. It makes it very hard to find something that I remember having seen about three months ago, or something I saw last year, etc. You get way too many results.
posted by alms at 9:42 AM on December 31, 2005

Pretty please.
posted by homunculus at 12:50 PM on December 31, 2005

To summarize.

Old search = good. New search = bad.
Cup of soup = good. Coldfusion = bad.

Old search - coldfusion + soup = Life complete.
posted by drpynchon at 2:21 PM on December 31, 2005

Why must you minus soup!!
posted by TwelveTwo at 3:29 AM on January 1, 2006

Ah, the good old days of MS-DOS! ..subject.. would bring up the subject. Why didn't Bill keep that feature in Windows?
posted by Cranberry at 12:49 PM on January 1, 2006

I still think that using Google Sitemaps would be the best solution and the easiest to implement.
posted by Sharcho at 3:02 PM on January 1, 2006

Even though Google's index of the site is horrible? Are you serious? An internal search engine would be best (if it didn't crash the server, natch), but Yahoo's indexing of MeFi is *much* better than Google's. We've established this over and over here, Sharcho, but feel free to compare a few site:metafilter.com searches at each one to see for yourself.
posted by mediareport at 3:10 PM on January 1, 2006

It's horrible at the moment. With Google Sitemaps the index will be complete and fresh.
posted by Sharcho at 4:45 PM on January 1, 2006

Really? Huh. If they can do that, then why are their normal MeFi search results so ridiculously spotty?
posted by mediareport at 5:19 PM on January 1, 2006

With Google Sitemaps you get:
* Better crawl coverage and fresher search results to help people find more of your web pages.

Most crawlers set limits on how much they crawl in order not to bog down the servers. Currently it spends most of the crawling resources on pages that haven't been changed.

Since it should be fairly straightforward to implement (it's basically just a generating a simple XML file), and since almost everyone uses Google, and since almost every built in search engine would be limited (e.g. no boolean searches, etc.) I really don't see why not.
posted by Sharcho at 4:00 PM on January 2, 2006

Currently it spends most of the crawling resources on pages that haven't been changed.

If that's true, I ask again, why is Google's index of *years-old* pages at MeFi so ridiculously spotty?
posted by mediareport at 4:53 PM on January 2, 2006

maybe google doesn't like the site structure. i would guess that they need to decde which pages to index and which not, and do so based on some heuristics. now it's quite possible that sitemaps they use different heuristics (maybe assume that anyone who goes the trouble of using sitemaps has more interesting data, for example, or perhaps there's some additional information/structure from sitemaps that changes how the pages are assessed, or perhaps as an incentive to encourage places like mefi to use sitemaps).

anyway, i think "it's complicated" is as good an answer as you're going to get. given that, for the reasons above, using sitemaps might make a difference. do you have a better explanation for why google isn't doing better, or why sitemaps would not help?
posted by andrew cooke at 6:52 AM on January 3, 2006

Maybe the crawler got JRun errors, and decided to give up on that certain URL, I really don't know. The good thing about Google Sitemaps is that when you sign up it tells you the crawl status, what URLs haven't been crawled and why.
posted by Sharcho at 3:51 PM on January 3, 2006

anyway, i think "it's complicated" is as good an answer as you're going to get.

Well, if that's all you have, guess I'll have to take it. I don't have any explanation myself; I'm just asking questions that seem to me to need an answer or two.
posted by mediareport at 9:50 PM on January 4, 2006

i see your point. i was wondering about this a bit more myself. it's depressing that it has (apparently, imho) got to the point where google is a black box.
posted by andrew cooke at 7:07 AM on January 5, 2006