robots exclusion suggestion for deleted threads February 22, 2006 8:35 AM   Subscribe

Just a followup to the Optical4Less spam and the robots exclusion suggestion for deleted threads: deleted AskMefi threads still do not have the exclusion meta tag, so that Mefi spam there can still be indexed by search engines. Anyone else spotted any other loopholes spammers might have slipped through?
posted by brownpau to Feature Requests at 8:35 AM (8 comments total)

done.
posted by mathowie (staff) at 9:14 AM on February 22, 2006


Technically, a user can just put spam links on their user page as well, correct? So do we have to exclude user pages from search engines as well? I'm just wondering how ultimately effective this will be: how much traffic is generated in searches for deleted pages anyway? And wouldn't it be BETTER for searches for spam products link to a page where everyone's ragging on the product and its provider for self-linking as a sort of punishment?
posted by XQUZYPHYR at 10:24 AM on February 22, 2006


So do we have to exclude user pages from search engines as well?

Been that way for ages.
posted by mendel at 10:51 AM on February 22, 2006


I kind of have a problem with the user page thing actually, one link per user seems like a very reasonable thing.

My understanding is that user page URLs were blocked because some users were receiving commercial offers to add links in the customizable space. I don't suppose there is any way to allow some links on a page to be indexed, but not others... Perhaps a page generated from the homepage URL field of each user?
posted by Chuckles at 3:13 PM on February 22, 2006


one link per user seems like a very reasonable thing

Some users have multiple, essentially disjoint sites.
posted by cortex at 3:52 PM on February 22, 2006


I kind of have a problem with the user page thing actually, one link per user seems like a very reasonable thing.

Users can (and do) still have links! They're just not indexed by search engines. Why should they be? The point of adding links to profiles is for other users to see them, not to increase pagerank.
posted by mendel at 4:52 PM on February 22, 2006


Exactly.
posted by cortex at 9:13 AM on February 23, 2006


If you remove all the legitimate links, all a search engine is left to index is spam...
posted by Chuckles at 10:43 AM on February 23, 2006


« Older Fix My FPP Links   |   Close Signups, Please Newer »

You are not logged in, either login or create an account to post comments