RSS feeds in Google searches August 1, 2007 1:39 AM   Subscribe

The new RSS feeds (which are wonderful) are starting to show up in Google searches. I doubt this is what anyone wants. I would suggest that they be disallowed in robots.txt, or set NOINDEX, NOFOLLOW.
posted by grouse to Bugs at 1:39 AM (14 comments total)

Double.
posted by WCityMike at 5:25 AM on August 1, 2007


Bad robot.
posted by liquorice at 5:34 AM on August 1, 2007


Now that there are a lot more feeds, it comes up a lot more.
posted by grouse at 6:08 AM on August 1, 2007


*nods furiously*
posted by Plutor at 6:24 AM on August 1, 2007


Indeed, the results are a real mess.
posted by Alvy Ampersand at 6:42 AM on August 1, 2007


Another option would be if the referral is a google search, it automatically switches to normal format. Or is that a bad thing?
posted by Deathalicious at 6:51 AM on August 1, 2007


Yes, it still screws up the results to have twice as many pages. And that kind of referrer sniffing doesn't seem like a good idea.
posted by grouse at 7:04 AM on August 1, 2007


From the other thread, I think it could be solved with a:

User-agent: Googlebot
Disallow: /*rss$


in the robots.txt file. Although robots.txt doesn't allow wildcards, Googlebot does.
posted by WCityMike at 8:04 AM on August 1, 2007


I'll try the RSS wildcard thing now...
posted by mathowie (staff) at 8:15 AM on August 1, 2007


ok, all robots.txt files have been updated. Let's see if that does the trick.
posted by mathowie (staff) at 8:19 AM on August 1, 2007


While we're at it, can we block off favorites pages? I hate it when that shit gets in the way when I'm a-hunting.
posted by cortex (staff) at 8:19 AM on August 1, 2007


Until MeFi RSS is out of Google's index, you could try sharpening your queries a bit. Try adding -inurl rss or -inurl favorites to exclude some pages.
posted by pb (staff) at 8:41 AM on August 1, 2007


And, you can create an alias in Dave's Quick Search Deskbar to insert the search refining strings automatically..
I love pimping for Dave. Unfortunately, my Aliases are completely out of date.
posted by Chuckles at 9:00 AM on August 1, 2007


Why not just add the parameters to the search form on the page? There's nothing you can do about stuff that's already been indexed except wait (and wait, and wait). But for the users of MeFi, it would be nice if the search box had the exclusion parameters hard-coded in hidden fields on the search form.
posted by Civil_Disobedient at 5:23 AM on August 2, 2007 [1 favorite]


« Older Double double?   |   del.icio.us profiles Newer »

You are not logged in, either login or create an account to post comments