3 posts tagged with crawl.
Displaying 1 through 3 of 3. Subscribe:

the rape of Google

An unfounded rumor about Carl Jung is now on the front page of Google, thanks to Ask Metafilter. [more inside]
posted by ikkyu2 on Sep 14, 2008 - 38 comments

Faster crawl rate option

Google is offering an option for a faster crawl rate
posted by Sharcho on Oct 22, 2006 - 8 comments

DNS wildcard and google crawler

It's possible that DNS wildcards on metafilter.com are causing problems for the Google crawler. I suggest deleting the redundant DNS entries, and using a 301 redirect for http://metafilter.com/ to go to http://www.metafilter.com/
posted by Sharcho on Jan 6, 2006 - 18 comments

Page: 1