Userpages closed to indexing?

January 23, 2005 2:01 PM

Revisiting this.

Metafilter is the biggest site that I can edit a page (my user page) on. I'd like to have "I am the [my name] that went to so and so school, and so and so high-school and did so-and-so" so that if someone googled my name looking for me, they could find that. However, userpages are marked NOINDEX for robots, for to stop villians from using google to harvest stuff, I assume. Given that it would be trivial to write a script to go through all the user pages and snarf anything, and given that these are The Internets, is there a reasonable expectation that an email address or other information you put on a publicly viewable page will *not* be harvested? Doesn't everyone have a spam-catcher address for this purpose?
posted by Capn to MetaFilter-related at 2:01 PM (7 comments total)

There's something related on the wiki that I didn't find until after posting.
posted by Capn at 2:14 PM on January 23, 2005

As long as it's on the web somewhere, and hyperlinked from somewhere else, Google will find it. Unless you have a very common name, having the info on your Mefi user page won't make much of a difference.
posted by Saucy Intruder at 2:32 PM on January 23, 2005

It's right that the userpages aren't cached as Metafilter has quite a high pagerank. It'd be trivial to fill your userpage up with links to viagra sites, and if the pages were indexed, only google would notice.

The robots.txt setting should stay. If you want people to find you on the internet, then (to use a common phrase), "get your own fucking blog"

More importantly, I think that the "comments by users" pages should also not be indexed. Every time I try and find anything on metafilter using google it always leads me to one of these pages. This must (a) impact the server performance, and (b) isn't a satisfactory result for a search.
posted by seanyboy at 2:40 PM on January 23, 2005

Slightly off-topic, but inspired by the link to the wiki - how often is the [links.mefi] page compiled? I'm on there, but with an old URL that I removed from my profile around a year ago.

As long as it's on the web somewhere, and hyperlinked from somewhere else, Google will find it
After 11 months of waiting, I'm not convinced this is 100% true.
posted by nylon at 2:45 PM on January 23, 2005

nylon, you're not trying hard enough.
I bet if I linked to your site, you'd be there in a month.
posted by seanyboy at 2:50 PM on January 23, 2005

I'm actually pretty glad that Googling my real name doesn't find my Metafilter user page. Please leave it that way.
posted by nicwolff at 8:00 PM on January 23, 2005

Agreeing with seanyboy, I find the "comments by users" pages every time I google+metafilter something and it's not very good, I mean I do find the thread I'm looking for eventually by scrolling that page, it would just be neater to find the thread at once instead. ;)
posted by dabitch at 3:21 AM on January 24, 2005