SpamBlocker 3.2.4-RC now ready for testing

Status
Not open for further replies.

nobaloney

NoBaloney Internet Svcs - In Memoriam †
Joined
Jun 16, 2003
Messages
26,113
Location
California
SpamBlocker 3.2.4-RC is now ready for testing.

We've reworked the code to block backscatter spam, and we'd like some testing. Don't forget to check all the EDIT locations in the file to make sure it works in your environment.

We've found some issues that I can't resolve with help from the DirectAdmin staff, unless YOU know of a workaround.

We use the domain names in in /etc/virtual/domains to determine if a domain sends mail from the server. The problem is that DirectAdmin uses the same file to determine if a domain receives mail on the server.

The problem is that when you change MX to a different server DirectAdmin removes the domain name from this file.

This has two effects, one which has always occurred since my first SpamBlocker technology exim.conf file; it simply makes spamblocking less effective:

Presuming you use /etc/virtual/use_rbl_domains as a link to /etc/virtual/domains, then spamblocking simply doesn't occur when senders ignore MX records and send email to your server because it hosts the A record for a domain. Some spammers do that, and I'd love to have a separate file of all domains on the server to use.

The problem in the latest file, 3.2.4-RC, is in addition to the above problem, which still remains.

If your server sends mail for a domain (for example, example.com) for which you've changed the MX server, then if some of that outgoing email from your server is to bad addresses, and Mailer-Daemon sends back a message, the sender will never get it, because the domain name isn't listed in the /etc/virtual/domains file.

The problem with that is that if you're using a list server of some kind, or even a form, to send email, and you're getting undeliverable replies from Mailer-Daemon, you need to get them, so you can take action.

To fix this, DirectAdmin would have to maintain for us a file with all domains on the server, whether or not they receive email.

Hopefully the DirectAdmin staff can help us with this, or suggest something else.

Yet another issue:

It's been brought to my attention that the code we use to check for legal Helo FQDNs is no longer accurate on today's Internet and will block mail to domains with numbers in their names, as well as mail to anyone using IDNs (International Domain Names).

Unless someone can provide us with code we can use, I'll probably remove the current code by the final release.

This is also something we'd like some testing on.

Your input is appreciated.

The new file may be found here.

Please try it and let me know your experiences.

Thanks.

Jeff
 
Hi Jeff,

The /etc/virtual/domains file is actually used to determine if a domain should recieve email. When the domain is removed from the /etc/virtual/domains file, this tells exim that the domain doesn't live here, so forces it to do the MX lookup and relay the mail away (with proper authentication of course).

So if the mail is from a remote box through port 25 to that domain, but the domain is removed from the domains file, then exim will need to relay it to the MX values. This would only be allowed if smtp-auth is being used, or if the IP is in the pophosts file.

If the mail is generated on the same box, the exim just looks up the MX record and sends it out, no auth required (assuming send with /usr/sbin/sendmail)

When the domain does exist in the /etc/virtual/domains file, then exim will look for /etc/virtual/domain.com/passwd or /etc/virtual/domain.com/aliases, etc.. (and other areas) .. to try and deliver it locally.


If you want a list of domains that exist on "this" box, no matter if the mail is supposed to be handled locally or not, you can use the:

/etc/virtual/domainowners

file, and just ignore the username that goes with the domain.

Thanks Jeff,

John
 
Unfortunately that won't work, John.

The problem is there's no way in exim.conf to tell exim to only use one field out of a file.

I need a file with a list of domains and nothing else.

I've asked the question on the exim-users list, and the response is that I need a list of all domains on the server which could send email.

I can figure out how often I should run a script to parse domainowners and create a new list, to satisfy me on my own servers, but I can't offer a public exim.conf file that requires a cronjob to be run.

If there's no way you can have DirectAdmin create a list, and if there's no suggestion you or anyone else can give me as to how to do this without requiring an additional script to run in order to use the SpamBlocker exim.conf file, then I'm going to leave it out of the final SpamBlocker 3.2. In that case my public exim.com SpamBlocker file will never be able to block collateral spam.

:(

Please see if you can help me by giving me such a file built into DirectAdmin, so we can implement control of collateral backscatter spam.

Thanks!

Jeff
 
Have you explored the possibility of writing a perl call for this? Perl can parse it out no problem.

Like

"${perl{getalldomains}}"

or:

"${perl{isthisdomainlocal}}"

etc.. then just tag on that function to the /etc/exim.pl

John
 
Actually, even simpler is to check to see if /etc/virtual/domain.com exists. If it does then the domain has data here. No parsing, no reading files, just check if the path exists and you'll know if the domain has local data regardless of MX records and contents of the "domains" file.

John
 
John,

Please note that I've just spent about an hour researching both in my hard-copy exim book (written by Dr. Hazel, who's head of the exim project), and in exim online documentation...
Have you explored the possibility of writing a perl call for this? Perl can parse it out no problem.
I've seen no way to do this in in a domainlist. There may be some other way to write a condition, but (a) I don't know how to write it; I'm an administrator and an analyst, not a programmer, and (b) if I did know how to write it, I most likely still wouldn't want to put the overhead of running yet another perl program on every incoming email into the exim.conf file.
Actually, even simpler is to check to see if /etc/virtual/domain.com exists. If it does then the domain has data here. No parsing, no reading files, just check if the path exists and you'll know if the domain has local data regardless of MX records and contents of the "domains" file.
Seems like a much better idea. Anyone care to write the condition for me, to put into the acl? Note that I'm actually looking for it to NOT exist.

If I can get help on this, I can add it to the new SpamBlocker, but I need it soon :).

Thanks.

Jeff
 
If there isn't any other way, it's really not difficult at all for me to generate a new file.

John
 
Sorry for the test message; I've deleted it.

John, you're the only person I know who knows how to write conditions for exim.conf; at least it looks that way, there are some conditions in the exim.conf file which appear to have been written by you.

I'd say, what's easiest for you. Or even, what's quickest for you, as all I'm trying to do is block collateral backscatter spam as quickly as possible, and that means getting it into the SpamBlocker3 tree as quickly as possible :). After SpamBlocker3.2 is released there most likely won't be any new features for a while.

Jeff
 
Can you paste me the context in which you are trying to use it? (small chunk of exim code) Then just put in a brief "english" condition, and I'll see what I can do from there to replace english with eximish.

John
 
Certainly, John.

The code currently in RC-4 is as follows:
Code:
# RC 3.2.4  09-nov-2009
  # Mailer-Daemon messages must be for us
  deny senders = :
       message = We don't host the recipient domain
       hosts   = !+relay_hosts
       domains = !+local_domains
       !authenticated = *
The problem is that I can't use local_domains as currently defined. I can create a new domainlist based on a new file you include in DirectAdmin, or I can rewrite it as follows:
Code:
# RC 3.2.4
  # Mailer-Daemon messages must be for us:
  deny message = We don't host the recipient domain
       hosts   = !+relay_hosts
       condition = "if recipient domain not in list \
                   of all domains on server"
       !authenticated = *
where the condition needs to be replaced with something you write which will get a list
of all domain names on the server.

Doing it the first way, mailer-daemon messages for true undeliverables never get back to the sender; doing it the second way will because exim will accept the messages and then relay them to the domain's real mx server.

By now I'm pretty good at eximish and I've got a good book besides :), but I've never learned how to write conditions, so I need your help.

Thanks!

Jeff
 
Hi Jeff,

I believe it would be something like:
Code:
condition = "${if exists{/etc/virtual/${domain}}{no}{yes}}"
where the negation is done by flipping the no and yes around. So if the path exists, the condition returns no (false) so the acl fails and isn't used.

John
 
@ Jeff
Did you check there?
http://wiki.exim.org/AclHeloTricks

Here is the HELO ACL I'm testing, based on that wiki page and formated to look like spamblocker code.
This is the whole acl_check_helo
Code:
acl_check_helo:
  # accept email originating on this server unconditionally
  accept  hosts = @[] : @

  # deny if the HELO is an IP address
  drop message = HELO is an IP address (See RFC2821 4.1.3)
      condition   = ${if isip{$sender_helo_name}}

  # deny if the HELO pretends to be this host
  drop message = Bad HELO - Host impersonating [$sender_helo_name] R1
       condition = ${if or { \
                          {match{$sender_helo_name}{$smtp_active_hostname}} \
                          {eq{$sender_helo_name}{[$interface_address]}} \
                           } {true}{false} }

  # deny if the HELO pretends to be one of the domains hosted on the server
  drop message = Bad HELO - Host impersonating [$sender_helo_name] R2
        condition = ${if match_domain{$sender_helo_name}{+local_domains}{true}{false}}
        hosts = ! +relay_from_hosts

  accept
This is to add in the acl_check_recipient acl because we need to skip authenticated users
Code:
  # deny if the HELO is neither a FQDN nor an address literal
  drop message = HELO should be a Fully Qualified Domain Name or an address literal (See RFC2821 4.1.1.1) R1
      !authenticated = *
      condition   = ${if match{$sender_helo_name}{\N^\[\N}{no}{yes}}
      condition   = ${if match{$sender_helo_name}{\N\.\N}{no}{yes}}

  drop message = HELO should be a Fully Qualified Domain Name or an address literal (See RFC2821 4.1.1.1) R2
      !authenticated = *
      condition   = ${if match{$sender_helo_name}{\N\.$\N}}

  drop message = HELO should be a Fully Qualified Domain Name or an address literal (See RFC2821 4.1.1.1) R3
      !authenticated = *
      condition   = ${if match{$sender_helo_name}{\N\.\.\N}}

I've removed this from Spamblocker:
Code:
  # RC 3.2.3  05-sep-2009 deny all unauthenticated if Helo not FQDN
  deny hosts   = !+relay_hosts 
       message = HELO should be Fully Qualified Domain Name  Host.Domain.Tld  See RFC821
       !authenticated = *
       condition =  ${if !match\
                     {$sender_helo_name}\
                     {\N.*[A-Za-z].*\..*[A-Za-z].*\N}\
                     {yes}{no}}

So far, I see a lot of RFC2821 4.1.1.1 compliant messages being rejected. I need to do more testing with exceptions to be able to tell if it's working a 100%.

Edit: Uncommented the domain checker
Edit2: Split the checks in 2 ACLs in order to be nice to Outlook clients...
 
Last edited:
What do you guys think of a parameter like this
delay = ${eval: ($rcpt_fail_count) * 60}s
For failed messages?

Would it work, or would it not matter because the next message from the spammer will be sent from a different host?
 
You've given me some stuff to study and hopefully work on during the weekend.

Note that here in the US the weekend starts tonight.

So this reply is only for your last post.

Where would you put that and why? We use what we call nolisting; looking it up on these forums should find the information on what it does and why we do it.

I fear your code may actually increase resource use on your server as well as tying up the spammer, but generally most spammers try once and move on, which is why nolisting has been somewhat successful.

Or am I missing something?

Thanks.

Jeff
 
That line of code that you can add in any rule asks exim to wait a certain number of seconds before giving the error message. Most spambots don't like to wait and will disconnect.
 
Wouldn't it also hold up exim until the disconnect? Couldn't that hold up exim and use resources? Don't spambots that don't like to wait also leave if you're using nolisting, which uses no system resources?

Jeff
 
I can confirm that nolisting does a better job than delay.

Regarding the HELO script, I'd like to include a counter that would blacklist for 24h, IPs of hosts sending bad HELOs 3 consecutive times, because most of my logs are filled with such bad requests and the same IPs try to send over and over.

What do you guys think? Worth the time?
 
Suggestion: Make the link between /etc/virtual/domains and /etc/virtual/use_rbl_domains the default behavior and introduce an exception file.
We only remove a few domains that absolutely don't want to use RBLs
 
Status
Not open for further replies.
Back
Top