2 posts tagged with crawler.
Displaying 1 through 2 of 2. Subscribe:

DNS wildcard and google crawler

It's possible that DNS wildcards on metafilter.com are causing problems for the Google crawler. I suggest deleting the redundant DNS entries, and using a 301 redirect for http://metafilter.com/ to go to http://www.metafilter.com/
posted by Sharcho on Jan 6, 2006 - 18 comments

Who are they and why are they copying MeFi?

Weird one:
Whilst googling for "MetaFilter Network LLC" [I was wondering how long Mefi had been so incorporated] I came across this page [google cache - they don't need the traffic.]
It appends a whole thread from ask.me on the end of their plain old links page of mortgage brokers - including the source code.
posted by dash_slot- on Aug 27, 2005 - 12 comments

Page: 1