Greylisting Code? Anti-Dictionary-Attack code?

[nobaloney]

I haven't been able to get greylistd work on a CentOS server, even using an RPM prepared specifically for the version of CentOS we use, which is CentOS 4.4.

In order to use a version which requires a daemon we have to have a daemon we can make work under every OS Distribution on which DA works.

That's the only reason I'm considering the MySQL version.

Jeff

 

[Scott DeLeury]

I'll see where I can get with it on one of my DA servers (not saying I'm going to get it to work when you didn't ) but it's worth a shot

 

[nobaloney]

Thanks, Scott.

It has to be foolproof on every platform DA supports or we can't make it a standard part of DA.

For several years now I've been saying DA supports too many platforms; maybe I'm actually right .

Jeff

 

[Maniak]

My vote went for both, due to the amout of email that will have to be checked, I prefer to make a restriction by greylist.

The dictionnary will tweak a bit more exim, and it is not bad at all.

 

[nobaloney]

I've decided that neither greylisting nor anti-dictionary attack code is going into SpamBlocker3.

I'm going to put greylisting into the next verison of SpamBlocker after DA starts compiling exim with MySQL support.

Jeff

 

[Chrysalis]

That is a shame I have seen mail servers go down that rely on mysql due to the mysql overhead involved with the high number of connections associated with dictionary attacks.

Isnt this possible without mysql?

 

[nobaloney]

Yes. But finding something that works without problems on all OS Distributions on which DA must work has been problematic for me.

You won't have to run greylisting.

You can certainly create your own solution, but any solution I come up with must work on all versions of DA on all distributions.

Jeff

 

[floyd]

I have a question and I realize that I am probably the dumbest person here. I am trying to figure out what makes a spammer's server different than any other server. Why would the spammer's server give up after the first try? What makes my servers keep trying? Why wouldn't the spammer just configure their server to resend again to get around the grey listing?

 

[nobaloney]

Most spammer use specially build servers/deamons/worms/viruses to send email. They're specifically designed to not retry.

Why? Because if he didn't, as he sent out millions of spams, he'd be getting hundreds of thousands of undeliverable notices. He really can't be bothered with doing anything with them; it's much more efficient for the spammer to keep sending to new addresses rather than try to figure out what to do with refused email.

Jeff

 

[tdldp]

Interesting Point of view

Hi jeff,

I was reading with interest your poll and thread concerning greylisting, as we start having difficulties with too restrictive SBL rules, that result in loosing some mails...

Concerning greylisting, i've seen working here in france an interesting method, that i find more efficient than the idea you exposed (note : no criticism of my part, i just love SB, since version 1, but want just to give a point of view)...

Indeed, your idea is that SB will accept mails on second sending from a senders adress (nothing is said, if it's IP or email)... This is not considering that spambox, often resend mails 3-4 times before sending a new email.
Well on my box and domain, i get in 24 hours, up to 10 mails totally identical with same IP / senders adress, which is spam (pharmaceutics / stocks)... If i understand well principles of greylisting, it will bypass blacklisting as soon an adress is in grey list.... Well in this case i'll get spam in my greylist, because it will pass greylisting tests, with 10 mails sent in 24 hours...

What i have seen working, is another greylisting method, based on alternative mail / web authentification...
A non listed (neither whitelist / neither greylist) sender mails to server, and is detected as new sender... A message is sent back to senders adress, specifying that the message is currently on hold, and will be delivered once authentification is done... The email adress is placed on greylisting, accepting all new mails but on hold (not delivering messages yet). In the response mail, is joined a weblink, with a sender Id that needs to be clicked or pasted in browser to validate sending. Sender has 24 - 72 hours (user setted), to respond to email by clicking on the link (sending on a php page)...Once this validation is passed, sender is added to whitelist. Any new mails sent by the same email adress will be accepted by server, with no new authentification process. If no action is undertaken under time given, senders IP and/or adress is added to autoblacklist...
This principle is used in many companies here in france, and announces false positives at zero, and 99.9999 % spam efficiency.

This solution is in france a commercial application, and i am unable to give you more details as not using it (i don't know if it sql based, if it works inside a panel, if it is reproductible), but i find this idea interesting and if you were to do something in this way, i would think this would be the best method to block spam really efficiently... It is post rbl controls and therefore is i suppose usable in exim...

I'll be trying in a few weeks to integrate this by myself in exim, (once all the other work on fire is done) and will keep you up with what i'll have found...

Tdldp

 

[nobaloney]

Lots of companies sell what you describe. It's even got a name:

Challenge-Response

Using it makes you a spammer. And I can't add it to SpamBlocker because then you and I would be spammers.

To find out why I think so, read this wikipedia article.

Jeff

 

[floyd]

I agree with you 100% Jeff. I had started writing a reply post to this but decided it was too long. Now that it has been a couple of days I have calmed down some.

Challenge response systems cause way too much email traffic and I will not use them even when I am the one being challenged. The only ones that I have seen use them are people sending me email. I have never been challenged when I send the initial email. I have only been challenged on replies to emails. If somebody is not smart enough to whitelist me before they email me then they just will not get a response from me. Half the time I cannot even read the letters and numbers on the graphic. I hate them. I abhor them. I will not use them ever.

 

[tdldp]

Lots of companies sell what you describe. It's even got a name:

Challenge-Response

Using it makes you a spammer. And I can't add it to SpamBlocker because then you and I would be spammers.

To find out why I think so, read this wikipedia article.

Jeff

After reading your post, and the wiki link, i agree that it is maybe not best solution, yet it is interesting... I think it could be interesting after all solutions you deploy in exim... Personnaly, i'll give it a test, and let you know how it works out on our system...

 

[floyd]

If you use it you will lose mail because people like me will not answer the challenge.

 

[nobaloney]

Not to mention that many people who can will put your IP# into either SpamCop or SORBS.

SpamCop not so bad, because it comes off after a while. SORBS makes you donate money (up to $25 for each piece of spam they believe you've sent) before they'll remove you.

Jeff

 

[thor]

Jeff,

I know you, and many others here, hate Challenge Response / Whitelisting, but I have to say for me it has worked great for years. Filters and SA block legit mail and allow some garbage through. C/R does not do that except in a blue moon when a spammer falsely sticks in a legit from address with an auto-response activated (for BoxTrapper which does not use a CAPTCHA webpage, but just a "reply to this message" approach).

I have a cPanel server with BoxTrapper and a DA server. I MUCH prefer DA overall, but cannot move all my domains over due to the BoxTrapper reliance for some address I, and my clients, get slammed with spam on and must have some form of C/R. I wish DA had a solution as such.

I have to disagree with you that it turns me into a spammer, because 90% of the time, spammers use bogus "from" / "reply to" email address and the mail will die after a few failure attempts at delivery (thus the greylisting idea). Do some spammers stick some poor unsuspecting users "from" address in there sometimes, yes. And that is a flaw, agreed. But I don't think it makes me the spammer? It is just like a doorbell on a house with a locked door. Who are you at my door? A salesman? go away... If a few people say, I did not knock on your door, that is a small casualty of something we all abhor.

However, one element of whitelisting that is vital is an auto whitelist of outgoing addresses. If I send a mail to someone (To, CC or BCC) it MUST add those addresses to the whitelist. Otherwise, as Floyd stated, it is entirely rude to send a reply to someone and then have to answer a C/R.

I wish and hope DA would implement a blacklist that just blackholes the mail on the blacklist, a whitelist that everything gets through on the list, an auto whitelist for outgoing mails, and then either a greylist as described in this thread (if you are firmly opposed to C/R), or a C/R with a CATCHA web page.

Make it all optional on a PER ADDRESS basis so I, and my users, can configure their own settings.

And if someone sends me a message that I have never emailed, and they do not want to whitelist themselves, I can always release their email from the holding bin. (and adding this feature to the greylist functionality would allow those "got to get this mail NOW" through if the recipient goes to the web interface to check the queue.)

Adding a configuration to a greylisting feature that says release on attempt 2,3, or 4, would allow you to increase the number of tries needed to resend if a spammers starts resending emails to try and defeat the greylist...

Anyway, just my thoughts. I love Challenge Response, because it keeps my inbox clear, and I can always check my queue. But I agree autowhitelisting MUST be included...

I just widh DA had either greylisting or C/R, with the other features (queue checking, auto whitelisting, etc)

As far as my IP being listed as spam, it has not happened in the years I have used C/R...

For guidelines on how to best deploy C/R to overcome hurdles, from the wikipedia site, http://www.templetons.com/brad/spam/challengeresponse.html

 

[floyd]

Business mail filtering of any kind is unacceptable. Either accept the mail or reject the mail. There is far too much money at stake to risk losing even one legitimate email. I will spend the 10 minutes per day deleting the spam that gets by the blocklists.

 

[thor]

I was not speaking of business mail filtering. That is why is should be 100% optional PER ADDRESS. One would be silly to turn that on for a business address. I totally agree too much $ is at stake to do that on a business address. However, for personal addresses.....

Any maybe, a hybrid system would be good too... start out with the greylist function, and on the second delivery attempt, THEN issue the Challenge Response. To cut down on all the excess spam mail being sent challenges...

 

[nobaloney]

We will continue to disagree. Vehemently.

I will not add any kind of challenge response system to my SpamBlocker exim.conf file. If DA decides to use it they can make any changes to it they wish, but I won't use any changes that require challenge response.

I almost never respond to a challenge-response request.

Sending challenge-response requests will at some point get you listed in SORBS and other blocklists as well. SORBS charges money to remove you.

Don't forget that if you use form-to-email forms on your sites, then no forms will ever get to you.

more below...

I know you, and many others here, hate Challenge Response / Whitelisting, but I have to say for me it has worked great for years.

More correctly the entire anti-spam community hates Challenge/Response. If you use it you will eventually be bit by this fact.

Filters and SA block legit mail and allow some garbage through. C/R does not do that except in a blue moon when a spammer falsely sticks in a legit from address with an auto-response activated

Many spammers use real from/to addresses from the same domain; you've probably got those addresses whitelisted already.

I wish DA had a solution as such.

Then write one .

I have to disagree with you that it turns me into a spammer, because 90% of the time, spammers use bogus "from" / "reply to" email address and the mail will die after a few failure attempts at delivery (thus the greylisting idea). Do some spammers stick some poor unsuspecting users "from" address in there sometimes, yes. And that is a flaw, agreed.

Your numbers were probably correct a few years ago. Today most spam comes from infected machines. The bots on those machines always use real addresses on those machines as the from address on those spams. If everyone used Challenge/Response then those return addresses would get millions of responses.

But I don't think it makes me the spammer?

You can think what you want. If you send unsolicited email to me in response to spam that didn't come from me, you're a spammer, and will be treated as such. And I'm not in the minority.

It is just like a doorbell on a house with a locked door. Who are you at my door? A salesman? go away... If a few people say, I did not knock on your door, that is a small casualty of something we all abhor.

You don't really see anything wrong with that analogy? Oh well. I can't convince you then, because you're blind to what's really happening.

I wish and hope DA would implement a blacklist that just blackholes the mail on the blacklist, a whitelist that everything gets through on the list,

Both of these are installed now, and have been for some time, as part of the SpamBlocker exim.conf file currently installed in DirectAdmin.

an auto whitelist for outgoing mails,

A serverwide whitelist with automatic additions is going to get very large very fast. Would you recommend that it be implemented in MySQL or as a flat file? Either one could cause large server loads when hit with dictionary attack email, as whitelists have to be checked before blocklists, so all incoming email must first be run through the whitelist. I'm not sure this is viable for many of us.

and then either a greylist as described in this thread (if you are firmly opposed to C/R), or a C/R with a CATCHA web page.

I'm not going to add Challenge/Response; someone else can but I won't use it or support it. If you think that CAPTCHA works, then please explain how automated bots still manage to post spam on these forums.

Make it all optional on a PER ADDRESS basis so I, and my users, can configure their own settings.

DA would have to decide what to add to the Control Panel; I only write the exim.conf file which so far, anyway, DA has chosen to use. There are two sources for plugins controlling SpamBlocker; perhaps one of them would want to do what you're suggesting. Note that per-address checking makes for huge flat files, huge configuration files, or complex MySQL setups.

And if someone sends me a message that I have never emailed, and they do not want to whitelist themselves, I can always release their email from the holding bin.

How does it take less time to look at email in the holding bin than it does in the inbox?

(and adding this feature to the greylist functionality would allow those "got to get this mail NOW" through if the recipient goes to the web interface to check the queue.)

Here you've lost me. Greylisting happens before you know anything about the email, before you've been able to do anything with the mail. Are you saying we should Greylist the mail but accept it anyway? How would we do that?

Adding a configuration to a greylisting feature that says release on attempt 2,3, or 4, would allow you to increase the number of tries needed to resend if a spammers starts resending emails to try and defeat the greylist...

Okay, so now we're talking thousands of hits per minute (on a busy server) to a database or flatfile?

Anyway, just my thoughts. I love Challenge Response, because it keeps my inbox clear, and I can always check my queue.

Which takes as long as checking my inbox.

I just wish DA had either greylisting or C/R, with the other features (queue checking, auto whitelisting, etc)

I plan on adding greylisting to SpamBlocker at some point soon. The rest won't come from me but of course you can do it and offer it to the rest of us, just as I've written SpamBlocker and given it to the rest of us.

As far as my IP being listed as spam, it has not happened in the years I have used C/R...

Which certainly doesn't mean it won't.

Jeff

 

[smtalk]

jlasman, any release date set for the beta2?