Exim + Greylist + Mrtg

Hi all

I ran into problems with this.

I'm running DA 1.28.0 with Centos 4.

First up, I can't su to mail:

[root@mail greylistd-0.8.6]# su mail
This account is currently not available.


Secondly, I can't start greylistd:

[root@mail greylistd-0.8.6]# /usr/sbin/greylistd &
[1] 18623
[root@mail greylistd-0.8.6]# Could not bind/listen to socket /var/run/greylistd/socket: (98, 'Address already in use')


Anyone any ideas? I'm running MailScanner too if that's any help.

I've tried doing the su after stopping MailScanner and the 2 exim instances, but to no avail.

Could it be Python which is out of date? I'm on version 2.3.
 
1.31.5 is the current directadmin

True, but I've seen nothing in the change log to suggest fundamental changes. I'm loathe to update in case it completely breaks my box - it's processing tens of thousands of email messages each day (mainly spam, hence the desire to greylist!).
 
I can't help you with the greylistd; as I've mentioned previously I never could get greylisting to work.

However I may be able to help you with your first problem: I don't think you can su to a username unless the username has a default working shell assigned in the /etc/passwd file.

Jeff
 
Hi Jeff

Do you think it's worthwhile bothering updating directAdmin? Can't see how this will help me, to be honest.

I reckon greylisting is one of the ways to go and I'm surprised there is so little around that is documented and updated properly.

I'm about to test BarricadeMX from www.fsl.com next week on another server that's not running DA so I'll see how a commercial greylisting based implementation works.
 
Do you think it's worthwhile bothering updating directAdmin? Can't see how this will help me, to be honest.
We always update DirectAdmin on one server immediately, then unless we find issues (only twice or thrice in the history of DirectAdmin) we do other servers within a few days. DirectAdmin has a great track record.
I reckon greylisting is one of the ways to go and I'm surprised there is so little around that is documented and updated properly.
The biggest issue I've found is that exim isn't compiled for MySQL; something to do with a key that would require exim to be compiled on every server, and DirectAdmin staff have told us there are good reasons to not compile exim on every server.

I've not found a system using flat files instead of MySQL that just works, and I've given up because it appears to me that nolisting, which I'm using on our servers seems to do the same thing, and is extremely easy. Search these forums for nolisting.
I'm about to test BarricadeMX from www.fsl.com next week on another server that's not running DA so I'll see how a commercial greylisting based implementation works.
Please let us know how it works for you. So far the only thing I can say about it is that reading the website it appears to not use anything we don't. That doesn't mean it won't do it better; it very well may, because it's optimized for email. DirectAdmin servers are optimized to deliver web pages first; email is definitely a secondary function. (That will become more evident when the next version of SpamBlocker exim.conf file is finally released within a week or so.)

Jeff
 
Hi Jeff

Thanks for your tip about /etc/passwd - you're right, the mail account was set to /sbin/nologin.

I have now managed to get greylistd working. I used the greylistd_0.8.6-0.1.tar.gz version from http://packages.debian.org/unstable/mail/greylistd and followed instructions I found on this thread.

I made some changes to exim.conf other than those described in this thread. Main one was to reference the /etc/greylistd/whitelist-hosts whitelist file:

Code:
# --------------------------------------------------------------------
# Check greylisting status for this particular peer/sender/recipient.
#
# Note that we do not greylist messages with NULL sender, because
# sender callout verification would break (and we might not be able
# to send mail to a host that performs callouts).
#
defer
  message     = $sender_host_address is not yet authorized to deliver mail \
                from <$sender_address> to <$local_part@$domain>. \
                Please try later.
  log_message = greylisted.
  domains     = +local_domains : +relay_domains
  !senders    = : postmaster@*
  !hosts      = : +relay_hosts : \
                ${if exists {/etc/greylistd/whitelist-hosts}\
                            {/etc/greylistd/whitelist-hosts}{}}
  set acl_m9  = $sender_host_address $sender_address $local_part@$domain
  set acl_m9  = ${readsocket{/var/run/greylistd/socket}{$acl_m9}{5s}{}{}}
  condition   = ${if eq {$acl_m9}{grey}{true}{false}}
  delay       = 20s
# --------------------------------------------------------------------

It's important to decide where to put the greylist stuff in the exim.conf - it must be placed above the part that accepts mail for domains you host or else, obviously, it will never get called.

I've added the IP ranges I can find for Microsoft, Google, Yahoo, Facebook, etc to the whitelist-hosts file and will carry on updating it.

The greylistd install comes with some man pages (listed in greylistd-0.8.6/debian/manpages):
greylistd-0.8.6/doc/man1/greylist.1
greylistd-0.8.6/doc/man8/greylistd-setup-exim4.8
greylistd-0.8.6/doc/man8/greylistd.8

I just copied the .8 ones to /usr/share/man/man8 and the .1 one to /usr/share/man/man1 and voila, I can now type man greylist or man greylistd for some help.

I noticed that I had a user called greylistd (I've spent so long on this that I can't remember how it got there) so I decided to run greylistd under that account. It meant I didn't need to mess about changing the mail account.

Create the greylistd user/group and add the mail user to it I got this from here:
Code:
useradd greylistd 
groupadd greylistd 
usermod -g mail -G greylistd mail

I made greylistd the owner of all the greylist stuff, as per the top post in this thread.

Of more interest are the results. The domains I host all have a secondary MX record. I pay a 3rd party for this facility. Even though I believe they use sbl-xbl, it is obviously a loophole. Still, while I'm in the early stages then this suits me because I don't want to block messages in error.

To give you some idea of volumes, yesterday over 80,000 SMTP connections were made to my server. Of these, over 42,000 tried to send a message with the rest being "incomplete transaction"s. Yesterday was Saturday and my customers are all businesses so on a weekday this number would be significantly greater.

Today, as at 13:45 we're at over 44,000 connections so about the same.

Before I began my greylisting project I was using MailScanner/SpamAssassin to check all the mail and delete high scoring spam. If a server was listed by two of the big RBLs such as sbl-xbl, spamcop, etc it was enough to ensure mail was deleted. One customer, for example, received over 18,000 connections one day last week and my server relayed on 1,695.

The problem I had was that my server was running ten to the dozen scanning all this rubbish for viruses and spam. Also, the customer who gets 1,695 messages per day was unhappy.

So earlier in the week I turned on sbl-xbl checking on Exim and of the 80k connections from yesterday, 69k or 86% were blocked. That really helps reduce the load on my server.

I've kept the sbl-xbl and it runs before the greylisting. My stats look like this:

Statistics since Sat May 24 16:27:47 2008 (21 hours and 37 minutes ago)
-----------------------------------------------------------------------
1353 items, matching 1358 requests, are currently whitelisted
0 items, matching 0 requests, are currently blacklisted
6117 items, matching 6144 requests, are currently greylisted

Of 8135 items that were initially greylisted:
- 1353 ( 16.6%) became whitelisted
- 6782 ( 83.4%) expired from the greylist

Looking at what MailScanner is up to, on 23rd May it processed 37,700 messages. In the last 10 hours it has processed 1,025 messages. Of these, 865 were classified as spam, of which 594 were classified as high scoring spam and deleted.

The difference is massive.

Previously the cpu was running at over 85% constantly, now I'm seeing figures like 0.7% with spikes occurring as messages are scanned for viruses.

As for the greylistd whitelisting, I have about 1,378 whitelisted now and 932 of these are from my secondary MX. The secondary MX is currently being greylisted for each sender and recipient address that goes via this route. In due course I may get rid of the secondary MX but I need to be sure that important mail isn't being blocked first.

Phew! Long post.
 
So are you going to write one (from beginning to end) How-To for us :) ?

I believe you'd get the same results from nolisting, but unfortunately the only way to check is to implement and notice the difference. nolisting doesn't log anywhere because the hits are to some other server.

But if you write the How-To I'd be happy to give it a try (replace nolisting on the server hosting nobaloney.net) and see if I can tell the difference.

Jeff
 
Yes, once I'm happy with the implementation I'll do a How-To.

Once concern I have is greylisting of mail from domains with multiple mail servers. Over the weekend it's not been an issue (I think) because nothing important is being sent. Tomorrow when businesses are back at work then there could be issues.

I'm not sure whether nolisting will work. I read up about it here -
http://www.junkemailfilter.com/spam/how_it_works.html - as you suggested and it means pointing the lowest priority MX at an IP not running SMTP.

Now in my case the domains on my server have a backup mx that works properly, ie, will retry once greylisted.

I can see that it receives only a small minority of the mail hosted on my server. Today only around 4,300 connections have come in from the backup mx, we can assume the vast majority are spam. There were 120k connections from other servers, 18k were greylisted and 44k were stopped by SpamCop. There were 45k that unexpectedly disconnected. I changed from Spamhaus to SpamCop because Spamhaus has new rules about overuse etc. I know these figures don't all add up properly but this is me just using grep against the mainlog.

What concerns me about greylisting is that from what I can work out, the various implementations have been around for some years yet most of them are not being maintained and documentation is sparse. I think postgrey could be the exception.

If it's so effective why isn't it used more? I may find out tomorrow when my customers start calling!
 
Correction re. nolisting. I'm sure it works to some degree and if my greylisting proves a success then I'll be abandoning my current secondary MX and either use a "dummy" secondary MX or one that also points to my server.
 
Okay i've given it a shot and got it working. Partly as described in this thread.

I've got it working on my CentOS machine using the following steps (from first post):

Install Greylist:

Code:
# rpm -Uvh http://dl.atrpms.net/el5-i386/atrpms/stable/greylistd-0.8.3.2-8.0.el5.noarch.rpm

Now you need to change permissions:
chown -R mail.mail /etc/greylistd/
chown -R mail.mail /var/run/greylistd/
chown -R mail.mail /var/lib/greylistd/
chown -R mail.mail /usr/sbin/greylist*

Now edit /etc/init.d/greylistd and change the 'su' user to mail:

Code:
start() {
        # Start daemons.
        echo -n $"Starting greylistd: "
#      daemon --user mail /usr/sbin/greylistd
initlog $INITLOG_ARGS -c "su -s /bin/bash - mail -c \"/usr/sbin/greylistd > /dev/null 2>&1 &\""
        sleep 2

Now start it using service or /etc/init.d/greylistd start. Check with ps if the user is running under mail and that the listing of /var/lib/greylistd is like (not sure if this is done directly) Update: This is only done when the process is stopped or restarted. It dumps it triplets/states to these files:

-rw------- 1 mail mail 148 Jul 30 17:21 states
-rw------- 1 mail mail 78 Jul 30 17:18 triplets

Now I only added (as far as i can remember) the following lines:

find:

Code:
# accept if address is in a local domain as long as recipient can be verified
  accept  domains = +local_domains
          endpass
	message = "Unknown User"

Add before:

Code:
# Perform greylisting.

    defer message 	= $sender_host_address is greylisted. Please try again later.
	log_message 	= greylisted.
	domains 	= +relay_domains : +local_domains
	!senders 	= : postmaster@*
	!hosts          = : +relay_hosts : \
			    ${if exists {/etc/greylistd/whitelist-hosts}\
                	    {/etc/greylistd/whitelist-hosts}{}} : \
			    ${if exists {/etc/virtual/whitelist-hosts}\
                	    {/etc/virtual/whitelist-hosts}{}}
                	    
	verify         	= recipient/callout=5s,use_sender,defer_ok
	condition      	= ${readsocket{/var/run/greylistd/socket}\
	                   {--grey $sender_host_address $sender_address $local_part@$domain}{5s}{}{false}}

Restart exim and check the logs. Personally I would change the retry value in /etc/greylistd/config to something less than an hour.

Hopefully this can help someone ;)
 
Last edited:
Please keep us updated as to how well this works and if you run into any issues.

Perhaps it should be in the next SpamBlocker exim.conf file.

Jeff
 
I have just installed greylistd on a test server, it seems that it's working ok. I only have one email domain configured on that server, so I'll have to wait and see how it's going on.

CentOS 5.2
Directadmin latest version
Spamblocker 3.1beta
spamassassin 3.2.5
clamav
greylisting (installed following instructions fusionictnl 31-07-2008)

Keep you informed,
Fossie
 
Grey work very good from one week.
Statistic:
2356 items, matching 5209 requests, are currently whitelisted
0 items, matching 0 requests, are currently blacklisted
521 items, matching 522 requests, are currently greylisted

Of 33703 items that were initially greylisted:
- 3049 ( 9.0%) became whitelisted
- 30654 ( 91.0%) expired from the greylist
 
Yesterday I've update DirectAdmin to 1.32.3 and suddenly greylist don't work.
When I send mail for example from my network, Outlook can't send email and then I got answer
Code:
451 83.238.213.67 is greylisted. Please try again later.
This e-mail was send on the same DA server.
Others networks have the same problem.
I don't have on this host SMTP, because it's only my network, but all time it was working.
I understand that should ask my SMTP server not my network.
 
I was going to set this up as suggested by snk and realized there are some differences because I am using spamblocker 3. Can anyone make some suggestions on what modifications needs to be made for spamblocker 3?

I have the current greylisted installed I just need the changes for exim.conf.
 
Back
Top