Anti-leech script which only allowed image use on a few sites

December 14, 2005 2:41 PM

After getting 20 gigs of traffic leached from my personal site I set up an Anti-leech script which only allowed image use on a few sites like metafilter, sa, etc, but to help the RSS users around here you'll need to allow a few more [more inside]
posted by holloway to MetaFilter-related at 2:41 PM (22 comments total)

Here's the list of valid referrers so far...

projects.metafilter.com
metatalk.metafilter.com
metafilter.com
ask.metafilter.com
bloglines.com
miniagg
newsgator.com
joel.lotek.org
neoflux.net
readablog.com

Any more?
posted by holloway at 2:42 PM on December 14, 2005

Actually, Matt has it set up so that *.metafilter.com works like www.metafilter.com.

smackfu.metafilter.com
posted by smackfu at 2:50 PM on December 14, 2005

and that's relevant to the script how?
posted by quonsar at 3:32 PM on December 14, 2005

bukkake
posted by fire&wings at 3:32 PM on December 14, 2005

(the script could wildcard *.metafilter.com if it wanted to instead of being explicit about the prefixes)
posted by cortex at 3:36 PM on December 14, 2005

Weren't there a few other domains mefi was available on?
posted by holloway at 4:10 PM on December 14, 2005

This is why god created wildcards.
posted by odinsdream at 4:29 PM on December 14, 2005

metafilter.org resolves also.
posted by stavrosthewonderchicken at 4:49 PM on December 14, 2005

Weren't there a few other domains mefi was available on?

Yeah. Stymied corporate drones, reveal your secret backdoors!
posted by cortex at 4:52 PM on December 14, 2005

I'll use wildcards anyway to deal with the www thing, and that'll remove 4 entries. Radical.

What I'm asking though – as you'll see from the other 6 entries in that list – is what other domains I should include (where else is mefi content used that I should include, are there any other sites that are rss aggregators, or rss readers that fake a referrers, etc).
posted by holloway at 4:58 PM on December 14, 2005

I don't get this. Do you want us to list all of the sites we like? Or all of the websites you like? Or do you want us to list all web-based RSS feeders?
posted by Plutor at 6:35 PM on December 14, 2005

Wouldn't you have better luck just posting an image in a Mefi thread and checking your webserver logs?
posted by boaz at 6:59 PM on December 14, 2005

Thanks Holloway.

metafilter.net ?
posted by If I Had An Anus at 7:12 PM on December 14, 2005

Other domains where metafilter content is used, and that I (or anyone) would need to allow images from.

Such as metafilter.org (thanks stavros), newsgator.com , etc.

And yeah, typically they'd be get their content via RSS, so I'll include those ones mentioned on wikipedia list, ta!

Boaz: This one is where the initial list came from. It seems that those referrers would be a bit of a random sample though, and maybe limited further by those who wanted to read the [more inside] about DIY projectors.

I'm not looking to get a perfect list but if there's any main sites that I've missed I'd like to hear about it :)
posted by holloway at 7:12 PM on December 14, 2005

Do standalone RSS readers that display HTML send the referer correctly?
posted by smackfu at 7:41 PM on December 14, 2005

The referer "miniagg" is presumably just an advert which isn't correct (RFC 2616: 14.36 says it must be a URI). Others seem to get it right and list the RSS feed's URL itself (which seems reasonable). I guess it depends.
posted by holloway at 7:54 PM on December 14, 2005

If you can use wildcards, you should just allow *

That should take care of most of the sites that metafilter content comes from.
posted by nomad at 8:24 PM on December 14, 2005

I own saving the world from evil dot com and am currently URL-redirecting to Metafilter, if any corporate drones are in the process of looking for a way to sneak around a firewall.
posted by onalark at 8:51 PM on December 14, 2005

That should take care of most of the sites that metafilter content comes from.

WHOOSH
posted by quonsar at 9:16 PM on December 14, 2005

I was waiting for that joke, nomad. Thing is that imaging the joke kinda takes the wind out of it, and then there's your joke, and, yeah.
posted by holloway at 9:41 PM on December 14, 2005

I imagined that response as much better than your real response too.

Maybe this is why everythings funny when you're stupid.
posted by nomad at 2:51 PM on December 15, 2005

Heh.
posted by holloway at 11:18 PM on December 15, 2005