How do I deal with spam when running my own email server?
June 1, 2004 10:02 AM Subscribe
SpamAssassin is, like, not working. I am dying under the weight of spam. I lie awake at night, crying, wounded. I love my e-mail address. I run my own e-mail server (using qmail and vpopmail). What's a boy to do?
i get no spam. well, i see maybe one every few days. i use spamassassin (inc bayesian filtering based on old spam) and white/black lists. some details are at http://www.acooke.org/andrew/writing/email.html, although it is a little out of date (self link).
(in particular, choose the block lists with care. currently i use relays.ordb.org, bl.spamcop.net, sbl.spamhaus.org, pom.blitzed.org, dnsbl.ahbl.org and list.dsbl.org)
posted by andrew cooke at 10:23 AM on June 1, 2004
(in particular, choose the block lists with care. currently i use relays.ordb.org, bl.spamcop.net, sbl.spamhaus.org, pom.blitzed.org, dnsbl.ahbl.org and list.dsbl.org)
posted by andrew cooke at 10:23 AM on June 1, 2004
You can also put Popfile in the mix, works pretty well for me (it's a Bayes filter)
posted by zeoslap at 10:23 AM on June 1, 2004
posted by zeoslap at 10:23 AM on June 1, 2004
Just as a related question, and not to derail the thread, but is it illegal to harvest emails from DNS records? I thought I heard something to that effect, and I've been getting some email at my disposable email address I set up specifically to block that. Is this reportable to any authorities?
posted by Hackworth at 10:51 AM on June 1, 2004
posted by Hackworth at 10:51 AM on June 1, 2004
I love knowspam.net but honestly can't recommend them, because there have be virtually no updates since the begining of this year.
posted by riffola at 11:20 AM on June 1, 2004
posted by riffola at 11:20 AM on June 1, 2004
Some of my geek friends are switching from SpamAssassin over to DSPAM.
posted by waxpancake at 11:21 AM on June 1, 2004
posted by waxpancake at 11:21 AM on June 1, 2004
My simple home-brewed procmail challenge-response system is blocking > 600 spams a day for me and very rarely letting one through.
posted by nicwolff at 11:42 AM on June 1, 2004
posted by nicwolff at 11:42 AM on June 1, 2004
I would take a good look at your SpamAssassin and MTA (if you run your own server) settings.
Make sure you are using the latest version of SpamAssassin. THIS IS A MUST.
Make sure Razor is being used.
Make sure Bayesian is turned on and working.
If you don't get a lot of valid messages from strangers, put your entire address book in as a whitelist, make everything in the whitelist equal a -8 points, then reduce the number of points to qualify as spam. 10 points is too high, 5 is average, 3 is hardcore.
Have your MTA verify the sending server's DNS claims. This one trick alone can cut 30% of spam because it's a good way to recognize forged headers, while still passing mail that is sent from a different host than that listed in the MX records for a domain.
posted by Mo Nickels at 11:46 AM on June 1, 2004
Make sure you are using the latest version of SpamAssassin. THIS IS A MUST.
Make sure Razor is being used.
Make sure Bayesian is turned on and working.
If you don't get a lot of valid messages from strangers, put your entire address book in as a whitelist, make everything in the whitelist equal a -8 points, then reduce the number of points to qualify as spam. 10 points is too high, 5 is average, 3 is hardcore.
Have your MTA verify the sending server's DNS claims. This one trick alone can cut 30% of spam because it's a good way to recognize forged headers, while still passing mail that is sent from a different host than that listed in the MX records for a domain.
posted by Mo Nickels at 11:46 AM on June 1, 2004
As someone who had been experiencing the same problem until very recently, I wholeheartedly recommend enabling SpamAssassin's Bayesian filtering. Spend the next few days storing as much spam into its own mailbox, train SA using the sa-learn command, and then commence the filterin', y'all. I've gone from ~60 messages per hour to maybe one per day slipping through, which then gets sa-learn'd for future recognition.
As Mo said, set your spam level at 5 and automatically procmail everything 5 or higher into a trash folder. I have yet to see a false positive.
posted by Danelope at 12:02 PM on June 1, 2004
As Mo said, set your spam level at 5 and automatically procmail everything 5 or higher into a trash folder. I have yet to see a false positive.
posted by Danelope at 12:02 PM on June 1, 2004
Danelope is right: teaching SpamAssassin about the spam mail it has missed using sa-learn is absolutely essential to keeping it running. I keep anything it misses in a "wildspam" folder and upload that to my server once a month to re-educate the filter. It's very, very easy: just FTP the mailbox and type "sa-learn --spam --mbox mailboxname.mbox" and you're done.
posted by bcwinters at 12:42 PM on June 1, 2004
posted by bcwinters at 12:42 PM on June 1, 2004
Thunderbird has a nice spam filter.
Have you had good success with Thunderbird's filtering? I've found it very limited. Then again, it seemed to work decently for a while but stopped. I suppose I could have mistrained it, but how many times does it need to see a drug name spelled out normally before it tags those as spam all the time?
posted by yerfatma at 1:01 PM on June 1, 2004
Have you had good success with Thunderbird's filtering? I've found it very limited. Then again, it seemed to work decently for a while but stopped. I suppose I could have mistrained it, but how many times does it need to see a drug name spelled out normally before it tags those as spam all the time?
posted by yerfatma at 1:01 PM on June 1, 2004
Training sa-learn is critical. Spammers know the default filters too well, so they just work around those. Whereas eeryone's corpus of spam is different, and that's why using sa-learn makes SA much harder for the spammers to beat. I get 80-90 spams per hour. With SA's out of the box settings, I was seeing >50 of those per day. After training sa-learn and tuning the config, only 3-5 per day slip through (that's because I like to keep the config conservative so I can catch false positives). That other 2000+ per day are all confirmed spam, and get cheerfully auto-killfiled without ever touching my InBox. It's a beautiful thing.
posted by nakedcodemonkey at 1:10 PM on June 1, 2004
posted by nakedcodemonkey at 1:10 PM on June 1, 2004
I have very good success with T-birds spam filters. Like 95% catches, and very few false positives.
It also quickly learns to filter out the different types virus-generated emails.
posted by signal at 1:22 PM on June 1, 2004
It also quickly learns to filter out the different types virus-generated emails.
posted by signal at 1:22 PM on June 1, 2004
Um, why don't you block all IPs that don't have a reverse DNS entry? Or are you doing this already? The majority of spams I get seem to originate not from a valid SMTP server at an ISP but from a DSL or cable modem connection. This should eliminate a lot of spam and pretty much all email viruses. Also do a check to make sure the sender domain is actually valid. I know Postfix lets you do this. Not sure about qmail. I highly recommend Postfix. I'm going to send Wietse a postcard any day now.
posted by estey at 1:29 PM on June 1, 2004
posted by estey at 1:29 PM on June 1, 2004
Response by poster: Oh, awesome. I had never even heard of sa-learn. I use my PowerBook 60% of the time and Mail.app gets spam nicely, but it still seeps through when I'm using squirrelmail/mutt, so I'll get sa-learn up and going.
AskMe rules. Thanks guys.
posted by xmutex at 1:42 PM on June 1, 2004
AskMe rules. Thanks guys.
posted by xmutex at 1:42 PM on June 1, 2004
My host supports SA, but it's a bit too much for me to install myself. Does anyone know of a service (or a person) who can help me with the install of SA?
posted by gen at 6:15 PM on June 1, 2004
posted by gen at 6:15 PM on June 1, 2004
OT gen are still using knowspam? Are you happy with them?
posted by riffola at 6:20 PM on June 1, 2004
posted by riffola at 6:20 PM on June 1, 2004
I did not use knowspam. I looked into it, but decided against. A few others I know use whitelists and they have had outtages and whatnot that I am not willing to deal with.
Mail.app is handling my spamfiltering right now, but I really want to kill the bulk of at the server than at my client.
posted by gen at 6:27 PM on June 1, 2004
Mail.app is handling my spamfiltering right now, but I really want to kill the bulk of at the server than at my client.
posted by gen at 6:27 PM on June 1, 2004
Oh ok sorry, I thought you did sign up for the trial back when you wrote about it.
posted by riffola at 7:09 PM on June 1, 2004
posted by riffola at 7:09 PM on June 1, 2004
Nic, that is sweet... I'm trying it now.
posted by Dean_Paxton at 9:11 PM on June 1, 2004
posted by Dean_Paxton at 9:11 PM on June 1, 2004
nicwolf ... Since my bogofilter database is fast approaching the size of my IP's kernel -- and STILL is letting way too much through -- I've been looking to do something much as you described. I've seen the general strategy before, but you're lucid explanation has pretty much sold me on this approach.
My only reservation is this: I'd love to see someone concoct a script through which outgoing mail could be piped that would automatically add the address to the white list. I can't think of any situation where I would deny email from someone I wrote to -- and this would go a long way to easing the (albeit small) burden I'm asking friends to endure. Granted, I can add it manually, but this would be a nice hack.
posted by RavinDave at 5:53 AM on June 2, 2004
My only reservation is this: I'd love to see someone concoct a script through which outgoing mail could be piped that would automatically add the address to the white list. I can't think of any situation where I would deny email from someone I wrote to -- and this would go a long way to easing the (albeit small) burden I'm asking friends to endure. Granted, I can add it manually, but this would be a nice hack.
posted by RavinDave at 5:53 AM on June 2, 2004
Upon reflection, nic ...
Wouldn't your method create an endless parade of bounced emails from fake spammer addys? And wouldn't those bounces, in turn, create a loop as each challenge to a fake addy is bounced back to your account? Wouldn't you need to do some sort of X-header sanity checking?
posted by RavinDave at 9:57 AM on June 3, 2004
Wouldn't your method create an endless parade of bounced emails from fake spammer addys? And wouldn't those bounces, in turn, create a loop as each challenge to a fake addy is bounced back to your account? Wouldn't you need to do some sort of X-header sanity checking?
posted by RavinDave at 9:57 AM on June 3, 2004
This thread is closed to new comments.
-Use layers of spam fighting -- I use spam assassin (always up to date) on my server to tag, and the outlook 2003 spam killer locally. Other combinations can work well too.
-Blackhole anything with <img or <IMG. Did wonders for my spam killing. People pretty much never do this, if you're on mailing lists or whatever, whitelist those addresses.
If that's still no good for you, you might have to consider one of those godawful places that make people who send you mail click a magic link to get added to your whitelist.
posted by malphigian at 10:17 AM on June 1, 2004