Greylisting Code? Anti-Dictionary-Attack code?

Should we add anti-dictionary-attack code or greylsting code to SpamBlocker3?

  • Add greylisting code

    Votes: 20 29.0%
  • Add anti-dictionary-attack code

    Votes: 8 11.6%
  • Add both

    Votes: 36 52.2%
  • Add neither

    Votes: 5 7.2%

  • Total voters
    69

nobaloney

NoBaloney Internet Svcs - In Memoriam †
Joined
Jun 16, 2003
Messages
26,113
Location
California
Should greylisting code, or code to block IP#s doing dictionary attacks, be added to SpamBlocker3 before the final release?

I can do one or both, but I really don't think the anti-dictionary-attack code is important if the greylisting code is added; I think the anti-dictionary-attack code would then be redundant.

What do you think?

Jeff
 
As long as these are options that can be turned on or off on a per domain basis.
 
this is useless ( always only my mind ;) )
i answered on other topics about that on the forum.

reject at smtp time is enough to do the job without heavy load.
 
Jeff,

Can you confirm greylisting ?

I understand it as..

Mail from a new email address is temporarily rejected at MTA level and the senders email address added to a database, then when/if the sending MTA attempts delivers a second time the email address is then moved onto a whitelist.

The principal being that spammers mail servers donot attempt resending mail (or even acknowledge failed deliveries)

If this is the case then I'm definately in favour of testing further BUT I am concerned with the size of the evergrowing database that would need to be kept on each server.

Say 1000 email addresses on each server, with an ever growing whitelist and greylist then I really feel that it could eat up resources cross referencing all the time. Is a 2 million record table easy enough to look up on quickly?

I maybe wrong on this (and would be happy to be wrong) - maybe you (Jeff) or someone has experience of this?

Maybe theres a way to limit table sizes (say email addresses are whitelisted for 30 days at a time) and the greylisted email addresses are given 12 hours to re-send before they are deleted from the greylist?

I also don't see how this would limit spam received from exploited php forms on senders servers and spam from servers that have been exploited and the existing server based MTA used to send spam. I'm also not sure what percentage of spam is sent using each method - have you done any tests or seen any sites that have these figures?


Rob
 
matrixx said:
Can you confirm greylisting ?

I understand it as..

Mail from a new email address is temporarily rejected at MTA level and the senders email address added to a database, then when/if the sending MTA attempts delivers a second time the email address is then moved onto a whitelist.

The principal being that spammers mail servers donot attempt resending mail (or even acknowledge failed deliveries)
Yes.
If this is the case then I'm definately in favour of testing further BUT I am concerned with the size of the evergrowing database that would need to be kept on each server.
I am, too, but I havent studied the issue other than to talk to others about it who says it works and saves a lot of server load. If there's interest I'll look into it. I'm probably going to implement it into our servers in any case.
Say 1000 email addresses on each server, with an ever growing whitelist and greylist then I really feel that it could eat up resources cross referencing all the time. Is a 2 million record table easy enough to look up on quickly?
I have no idea. Anyone?
I maybe wrong on this (and would be happy to be wrong) - maybe you (Jeff) or someone has experience of this?
Anyone?
Maybe theres a way to limit table sizes (say email addresses are whitelisted for 30 days at a time) and the greylisted email addresses are given 12 hours to re-send before they are deleted from the greylist?
I found two implementations for exim discussed here. Care to do the research for us, Rob :) ?
I also don't see how this would limit spam received from exploited php forms on senders servers and spam from servers that have been exploited and the existing server based MTA used to send spam.
Standard MTA servers will of course eventually lresend, but they're a very small amount of the spam on the 'net today. Most spam today comes from spambots, and this appears to be the best way to block them.
I'm also not sure what percentage of spam is sent using each method - have you done any tests or seen any sites that have these figures?
Yes, from various anti-spam lists and newsgroups. I'm not doing formal research and I haven't kept notes. I plan on eventually implementing for me, ahead of blocklists, and see how much lower blocklist use gets.

Jeff
 
The link looks ok - I'd definately be interested in this.

Would you plan to permanently add it pre-blocklist?

I can see that adding it pre-blocklist would give a gauge as to it's effectiveness.

Because blocklists filter 'definate' spammers I was wondering if (after testing) it would come after the definate spammers have been weeded out?

R
 
I've made no decisions :) . I haven't even decided yet if I'm going to continue to work on SpamBlocker; obviously there are people here who think they can do much better than me; perhaps it's time to let them try.

Jeff
 
Dear Jeff,

If no one else will say it, then I will.

Your work on SpamBlocker is TRULY appreciated by the silent majority of DA users, and we would be extremely saddened if you were to drop the project because there is but ONE vociferous complainer.

In Spanish, and attributed to Cervantes Saavedra, (though not true), we have a saying, "Los Perros Ladran, Sancho, señal que avanzamos", said by El Quijote to his trusty servant Sancho Panza. (The dogs bark which means we are making progress).

That is, we don't go unnoticed in life, because wherever we thread, we leave footprints and the dogs bark to signal our passage.

Let this dog bark, and pay it no attention.

I am willing to give Mr. X. the benefit of the doubt. He has provided some useful pointers, and I have even applied one of his recommendations to good effect. I would like to think, that he, like me, has trouble with a language that is not his native tongue and therefore expresses himself badly.

I'd say, from now on, just ignore his misdirected comments, and let the other THOUSANDS of grateful DA admins and users reward you with their sympathy and appreciation, even if it goes unspoken.

You, Mr. Lasman, hold the higher moral ground. Mr. X only serves to constantly remind us of that.

Do not stop work on the Spamblocker project. We know you are right, end of discussion.


Regards,


Carlos - PanamaSpace
Just one of the many GRATEFUL & SATISFIED SPAMBLOCKER DA users lurking on these boards.:eek:

p.s. I can't wait for Spamblocker 3 to go Final!!!
 
Jeff,

You know I love your work. But I will have to say that having a database that will grow and possibly bloat isn't a good idea. Bloating is going to cause greater resource usage and on email spikes the outcome can take down a machine. Heck with ClamAV and SA running the spikes already cause a few minutes of congestion from time to time.

I think that we should use a RBL dedicated to the greylisted IPs rather than having a database on every server running spamblocker. But of course the chore in that case is that someone would have to write a utility to propagate it. And of course someone has to cough up the cash and resources to host it. Xemaps? LOL!

In the case of the non-dictionary-attack using the right SA rules seems to block that already.

So my votes are in.......

Big Wil
 
BigWil,

I think you may misunderstand greylisting. You can't have a central blocklist for greylisting because greylisting is based entirely on email coming to your server. The first time it comes you tell it to go away and come back later.

When it comes back later you accept.

I most likely will include greylisting in the final release (I'm starting to study it tonight) because it has the possibility of taking a lot of load off the server. In fact I believe it's the easiest way to block most spam and if run before blocklists will probably cut their positives almost to zero (I'm looking forward to seeing if I'm right :) ).

I'm probably going to add anti-dictionary-attack code before the greylisting code because I believe it will keep the greylisting database smaller than it would otherwise be.

If possible both will be optional on a per-domain basis.

Jeff
 
No that is about what I understood it to be. But the info is stored until it comes back correct? So depending on the sending mail server's settings we are looking at several hours before email delivery is tried again yes? But if the receiving server gets say 300 email delivery attempts in a minute or even worse how bloated can a database get. Pretty darn bloated is my guess.

Secondly, what if a customer wants their email right away?

Any insight is greatly appreciated.

Big Wil
 
A few minutes to less than an hour, generally.

We won't ever know the load it will generate on your server. We're testing it on my server.

If we add greylisting you'll have to enable it on a per-domain basis, for the domains on your server. We recommend you offer greylisting to your clients but don't force them to use it.

Jeff
 
Ok. Well as long as it is a disable until enabled scenario then I think that will be fine. Have a Happy and Safe New Years.

Big Wil
 
I haven't been able to get it running yet; using the solution requiring a separate daemon I can't get the daemon to run on any of our servers :( . And I don't yet have a copy of exim compiled for mysql support.

So moving forward... but slowly.

Jeff
 
I was thinking, this grey-listing, will this happen to every email which is delivered to the mail server?

If so, then all the (real) mail servers would have to send the mail twice to get a succesfull answer right? (please correct me if i'm wrong).

If everybody (or a lot of mail servers) would use this grey listing, then wouldn't all the mail servers be very busy sending mails, checking for an answer and then resend it and then finally get his answer?

That would double the email activity?

Just some of my thoughts...

Niels
 
That depends on how long you keep the information in the database. Once in the database, all email will go through the first time.

What would you want to do if you could eliminate almost all the spam you now get by implementing a delay? Would you like it?

Jeff
 
I'm not quite sure, I can't see through all the pro's and cons.

If you'd ask me, would you like a solution for all(most) your spam, offcourse the answer would be yes.

A delay... I'm not quite sure about that... Part of me says yes, but I would however like the email to go through directly... But al this spam...

I understand your difficulty about making the choice ;(...

In my case it's just a private server hosting some domains for me and some friends... So it's not a very big problem for me. Maybe others with large production servers have some good ideas....

Greetings,
Niels
 
NSteffens said:
I'm not quite sure, I can't see through all the pro's and cons.
Neither DA staff nor I will ever propose a default solution that'll cost you problems or clients.
If you'd ask me, would you like a solution for all(most) your spam, offcourse the answer would be yes.

A delay... I'm not quite sure about that... Part of me says yes, but I would however like the email to go through directly... But al this spam...
Contrary to what anyone else may be saying, the anti-spam community consensus appears to be that greylisting is the best way to eliminate spammers today. The delay for most well-behaved servers is about five minutes. Some servers may delay up to an hour, and of course a few badly cofigured servers will give up immediately. While there's no such thing as a world without false positives these false positives will result in the email being returned to the sender with a notification that it didn't go through. Which is much better than solutions that (for example) blackhole the suspected email so no one ever sees it and senders believe you've got it. I don't like it any more than you do; I refused to consider it a few months ago when I was asked to do it.

Hovever times are changing rapidly, and today I believe it's worth the delay.

That said, it won't be in the release versioon of SpamBlocker3 because it requires a copy of exim with mysql support enabled. I can do that for my platform, CentOS, but I can't do it for every platform, so it's not being included.

When it is included it'll be configurable on a per-domain basis.
I understand your difficulty about making the choice ;(...

In my case it's just a private server hosting some domains for me and some friends... So it's not a very big problem for me. Maybe others with large production servers have some good ideas....

Actually for you it's an easier decision to use it than it is for those of us with shared hsoting servers for clients. Historically people with their own servers have always been more conservative about what they accept than those of us with commercial hosting services. We have to be liberal because we can't risk losing important customer emails.

I really hoped others would chime in constructively, but so far we've gotten only complaints but no better solutions :( .

I've got no ulterior motive at all in producting SpamBlocker for DA. Unless you call wanting to keep spam off my servers an ulterior motive.

Jeff
 
Personally, I agree that greylisting is a wonderful front-line defense to spam. Most spammers' servers will give up after the first try when an email is greylisted, while legitimate emails will be re-sent again after a specified period of time.

There are implementations that use MySQL and there are those that don't (here's one: http://blog.adamsweet.org/?p=172)

I've been looking into this for a bit on my own for work purposes on another MTA installation, and will be implementing it when I get back from vacation on Wed.

Greylisting is a legitimate and widely used anti-spam practice these days, so why not make it a 'standard' with DA? :)
 
Back
Top