Unhappy customers - lack of filtering and anti-spam

scooby2

Verified User
Joined
May 18, 2008
Messages
13
Location
Chicago, IL
Two issues that my customers are extremely unhappy about. Has anyone had any luck with either of these?

1. Everyone wants to know why they cannot have procmail, sieve, or some other kind of filtering on inbound email messages. I have had some minor luck with procmail but for some reason it double delivers messages that should goto the inbox. I feel DA is being left behind here.

2. Spam has been out of control for quite a while now. It would be nice to use an anti-spam program like dspam or something similar. spamassassin is pretty bad these days. Spamblocker 4 helps some but its not an anti-spam program.

Thanks,
Scooby2
 
Not me. I don't want to know.

Remember, you can use any 3rd party software with directadmin, if you're skilled enough to replace that one coming with directadmin from a box.
 
I'd be willing to bet you never installed SpamBlocker Technology (written by me) on your domains. Search these forums or hire me to install it (http://www.nobaloney.net/spamblocker.html).

You can install either procmail or seive, and rewrite exim.conf yourself to use them, or you can learn how to do the same thing with delivery options and filtering options already built into exim. How come none of us are having these problems and you are? Is it because you haven't taken time to learn how DirectAdmin does it?

Jeff
 
I'm definitely using Spamblocker 4 but as far as I can tell, its not doing bayesian or any kind of statistical calculations based on body content. Spamblocker seems to give Exim the equivalent of what Postfix gives me out of the box with minor tweaks.

As for filtering, DirectAdmin does not do it. I'm not sure what you are talking about but please enlighten me on how to have DirectAdmin filter (not spam filter) mailing lists and such so that imap users have everything nicely sorted into folders. I have a lot of blackberry customers that want their mail nicely sorted. They want to be able to sort into folders based on To/From address, Subject, and Body content. Gmail can do it. Why not DirectAdmin?

I'd be willing to bet you never installed SpamBlocker Technology (written by me) on your domains. Search these forums or hire me to install it (http://www.nobaloney.net/spamblocker.html).

You can install either procmail or seive, and rewrite exim.conf yourself to use them, or you can learn how to do the same thing with delivery options and filtering options already built into exim. How come none of us are having these problems and you are? Is it because you haven't taken time to learn how DirectAdmin does it?

Jeff
 
I'm definitely using Spamblocker 4 but as far as I can tell, its not doing bayesian or any kind of statistical calculations based on body content. Spamblocker seems to give Exim the equivalent of what Postfix gives me out of the box with minor tweaks.
SpamBlocker blocks based on certain behaviours and reputation. It's not designed to do any body checking; that can be done easily after email is accepted, by SpamAssassin.

I understand that you (and others) may want email bodies to be checked before email is accepted. There are several reasons why I didn't take this approach:

In my experience, I get such a small amount of email caught by SpamAssassin after using SpamBlocker that I didn't see the reason to put up with the overhead involved in checking bodies at receipt time.

Many senders don't understand a block after the body is accepted, and go ahead and retry the email later, over and over again, creating an even higher overhead. This less true than it was in the past; most spam is now sent by spambots, and generally they never retry. But this was part of my original decision.

SpamAssassin does it's job well, and in the DirectAdmin configuration, doesn't attempt to send back email to any senders, so it can't send email back to the wrong senders.

So, if I decide to take this approach, it won't be to fill my own need, but rather to create a product for others.

This product would run a SpamAssassin configuration, tweakable by you, but otherwise standard. You'd not be able to use most of the current SpamAssassin settings in DirectAdmin because except for the trigger value for blocking, they wouldn't make sense.

This product would not be a commercial product because it would use the exim.conf file, which couldn't be encrypted, and because exim is published under the GPL, must be open-source in any event.

So my interest in doing this would be based on advance subscription, with every subscriber understanding that once it's done, anyone would be able to use it, whether they subscribed or not.

Or I suppose I could open up a donations page for SpamBlocker, and see if there were enough donors now to pay for my time.

Or of course you could do it yourself, or find someone else who wanted it badly enough to do it.

My SpamBlocker exim.conf files (all versions) are all covered under the GPL, Version 2, so you can take anything I've done and add to it, or modify it.
As for filtering, DirectAdmin does not do it. I'm not sure what you are talking about but please enlighten me on how to have DirectAdmin filter (not spam filter) mailing lists and such so that imap users have everything nicely sorted into folders.
DirectAdmin uses both a system filter file and a perl file as part of it's exim configuration. In the SpamBlocker Version 4 exim.conf file for DirectAdmin see both Edit#4 and Edit#5. You can use these to do almost anything you might want.

Again, I've not had the need, and so I haven't written additional hooks into my files, but you, or others, most certainly can.
I have a lot of blackberry customers that want their mail nicely sorted. They want to be able to sort into folders based on To/From address, Subject, and Body content. Gmail can do it. Why not DirectAdmin?
GMail isn't an MTA (mail transfer agent [wikipedia.org]); it's an MUA (mail user agent [wikipedia.org]). Specifically a web-based MUA. DirectAdmin offers several of those, including Squirrel, AtMail, and others. And you can install any others. Some of these include filtering capability which can move emails into specific folders, and once they've been moved, they can be used by other imap-based programs. Alternatively, a desktop-based or laptop-based imap client can do the same. And once it's done, of course your blackberry could take advantage of it as well.

If your clients need more sophistication than that, of course you can offer them a premium email solution, as we do for our email clients who need it.

Further discussion is encouraged.

Jeff
 
Hello,

Also, it was mentioned that no Bayesian filtering is being done. That would be the job of SpamAssassin. It can be enabled with this guide:
http://help.directadmin.com/item.php?id=36

We use it, and it does a great job.

As for email redirecting into specific folders based on content, that's more of the scope of the email client. As Jeff mentioned, Squirrelmail, Roundcube, or even their outlook express (or whichever client they're using)

It is a possible to do that sort of redirecting in the exim filters (Eg: /etc/virtual/domain.com/filter), but it hasn't been something we've had requests for. Note that the filters are templates, and they can be customized however you wish. Per-user customizations using the basic scripting within them would allow for essentially anything you want, but at this time, it's not something could be done through the interface (unless someone made a plugin for it)


spamassassin is pretty bad these days
I'm not sure I would agree there, it might be a misconfiguration on your server. Make sure you're running the latest version of SpamAssassin, and run the command "sa-update" every few weeks to get the latest rules. Ours is set to flag spam at 5, and delete high scoring spam at 7. We don't get much spam. You can go one step farther by using the sa-learn program to help SA figure out what is and isn't spam. Also, the RBL blocking that SpamBlocker supports is very solid (but needs to be enabled, in case that has not been done yet)

John
 
I have shared hosting clients on my server. I have Spamassassin installed from before it was made available in Custombuild. Also it is version 2.x I currently have installed.

If I set Custombuild to use Spamassassin and rebuild it in Custombuild, will this create trouble for the existing Spamassassin version manually already installed? Will it be upgraded/overwritten by Custombuild? Also I am afraid if my users will loose their settings for Spamassassin and their domains if I convert to use Custombuild for Spamassassin ...

I don't think I have it uncommented in exim.conf either. I think this is bad, and it should have been. I will have to investigate ...

Edit: So, I guess my question is, if it is hard and problematic to convert Spamassassin from manuall install to custombuild install?
 
Hello,

If you have the latest version of DA, I would say to try and update to 3.3.1.
The custombuild version matches the spam.sh version, so doesn't make too much difference.

Note, that I've removed the install of the required modules (the "downloadMake" calls), before installing Spamassassin itself, since the number of prerequisites for those modules was getting difficult to maintain. Using cpan is far simpler, more reliable, and will get all dependencies needed.

It's outlined in the SpamAssassin install guide

The current spam.sh may still call "downloadMake", as well as custombuild. The next release of DA will have them removed, and for custombuild, I'll update it shortly to remove them as well.

The replacement command will be:
Code:
 cpan -i Archive::Tar Digest::SHA Mail::SPF IP::Country Net::Ident IO::Socket::INET6 Compress::Zlib Mail::DKIM LWP::UserAgent HTTP::Date Encode::Detect ExtUtils::MakeMaker
which you'd run before running spam.sh (or custombuild)

In your case, since you're running the very old 2.x version, depending on how old your OS is, you may run into problems if your perl version is too old (SA needs a certain perl version, it will tell you). I'd recommend trying anyway to see if it works.

In your case, run the above cpan command to get all perl modules installed.
If it asks to do manual config, answer "no" and it will configure it for you.
If it asks about pre-pending modules to the list, answer yes.

Since we don't want to run the downloadMake calls (as cpan will do it for us), edit the spam.sh, and change all calls of "downloadMake" to be "#downloadMake", so that it's commented out (except the function itself). Then make sure you have VERSION=3.3.1 at the top. Save/exit, then run it.


Read the guide for more info. Installing SpamAssassin won't do anything if you don't enable it in the exim.conf.


Installing custombuild version overtop of the same version from spam.sh (3.3.1) won't hurt it, but will also be pointless (unless you deleted something)

Installing SpamAssassin from either method will not affect your User settings (/home/user/.spamassassin/user_prefs). Both methods only install the spamd/spamc binaries, and their data.

There is no "convert" between the 2 methods. They both do the same thing with essentially the same code. They compile and install the binaries. Overwriting the binaries with the same thing again doesn't hurt it.

John
 
Hi John,

Thank you very much for your helpful answer! I just checked my Spamassassin version, and to my suprice I have 3.x version:

Code:
[root@server ~]# perl -MMail::SpamAssassin -e 'print $Mail::SpamAssassin::VERSION."\n";'
3.003001

Then I checked my /etc/exim.conf file, and found this:

Code:
# Spam Assassin
#spamcheck_director:
#  driver = accept
#  condition = "${if and { \
#			{!def:h_X-Spam-Flag:} \
#			{!eq {$received_protocol}{spam-scanned}} \
#			{!eq {$received_protocol}{local}} \
#			{exists{/home/${lookup{$domain}lsearch{/etc/virtual/domainowners}{$value}}/.spamassassin/user_prefs}} \
#			{<{$message_size}{500k}} \
#		} {1}{0}}"
#  retry_use_local_part
#  transport = spamcheck
#  no_verify

But longer down in that file I have this:

Code:
# Spam Assassin
begin transports

spamcheck:
  driver = pipe
  batch_max = 100
  command = /usr/sbin/exim -oMr spam-scanned -bS
  current_directory = "/tmp"
  group = mail
  home_directory = "/tmp"
  log_output
  message_prefix = 
  message_suffix = 
  return_fail_output
  no_return_path_add
  transport_filter = /usr/bin/spamc -u ${lookup{$domain}lsearch*{/etc/virtual/domainowners}{$value}}
  use_bsmtp
  user = mail
  # must use a privileged user to set $received_protocol on the way back in!

First of all, I am not sure Spamassassin currently is doing anything for users who activate it on their domains, because it might not be activated correct in exim.conf

Second I am not sure if I must remove and comment out the last block with Spamassassin code in exim.conf if I uncomment it higher up where it is commented out at the moment.

I would like to convert to use Custombuild for Spamassassin, so that I can more easy upgrade Spamassassin using custombuild in the future. I understand you that it will not be a problem. So I will try. I must first read you message over and over, and study this some more.

Spamassassin was previous installed by outsourced tech people. I guess I must learn to do it myself in order to get it correct. This is bad for me. Thank you for all the help!
 
Hi John,

Big question: I just noticed that my /etc/exim.conf file is a SpamBlocker file, first line in that file is:

SpamBlocker.exim.conf.2.1.1-release

But I have never installed Spamblocker (I want to later on). So this means that either one of my previous tech people did this, or that Custombuild automatically use this Spamblocker file for exim?

I have Exim version 4.73 installed using Custombuild. Does DirectAdmin/Exim automatically install this Spamblocker version of etc/exim.conf, is that why I have that content in that file?
 
Hello,

All you need to do is edit your exim.conf.
Change this code:
Code:
# Spam Assassin
#spamcheck_director:
#  driver = accept
#  condition = "${if and { \
#			{!def:h_X-Spam-Flag:} \
#			{!eq {$received_protocol}{spam-scanned}} \
#			{!eq {$received_protocol}{local}} \
#			{exists{/home/${lookup{$domain}lsearch{/etc/virtual/domainowners}{$value}}/.spamassassin/user_prefs}} \
#			{<{$message_size}{500k}} \
#		} {1}{0}}"
#  retry_use_local_part
#  transport = spamcheck
#  no_verify
to look like this, then restart exim
Code:
# Spam Assassin
spamcheck_director:
  driver = accept
  condition = "${if and { \
			{!def:h_X-Spam-Flag:} \
			{!eq {$received_protocol}{spam-scanned}} \
			{!eq {$received_protocol}{local}} \
			{exists{/home/${lookup{$domain}lsearch{/etc/virtual/domainowners}{$value}}/.spamassassin/user_prefs}} \
			{<{$message_size}{500k}} \
		} {1}{0}}"
  retry_use_local_part
  transport = spamcheck
  no_verify
Related:
http://help.directadmin.com/item.php?id=280

The 2.2.1 version of SpamBlocker is our default that comes with DA, that's fine.
The SpamAssassin part will be essentially the same regardless of SpamBlocker version.

The spam blocking with SpamBlocker kicks in when you enable RBLs (if I'm correct)
http://help.directadmin.com/item.php?id=142

SpamBlocker and SpamAssassin are 2 different things.
You can use both, but each does what it's designed to do very well, but they work differently.

John
 
[..]
The 2.2.1 version of SpamBlocker is our default that comes with DA, that's fine.

Thank you! A moment I was feeling that I was loosing control. But now I know that previous tech people have not done anything without telling me about it, but that this comes from DirectAdmin/Custombuild. What a relief.

[..]
The spam blocking with SpamBlocker kicks in when you enable RBLs (if I'm correct)
http://help.directadmin.com/item.php?id=142

SpamBlocker and SpamAssassin are 2 different things.
You can use both, but each does what it's designed to do very well, but they work differently.

Again, thanks. I am in a stressfull situation today, but will configure this soon. I might contact jlasman and ask if he is willing to take the job with installing and configuring newest SpamBlocker, and also convert Spamassassin to use custombuild and configure it properly. Spam is a problem for my customers, thats for sure.
 
@scooby2 - If you have business customers and/or your customers rely heavily on webmail, I definitely recommend you get rid of the whole vacation/autoresponder/filter system from DA and Exim and install Pigeonhole (sieve) for Dovecot.
If you setup things properly, email users will be able to have full control over server-side filters from the webmail and will never have to login into DA (why should they ever?).
With the right webmail, they would have access to full account information, they would be able to change their password and synchronize their data with mobile devices.

Regarding the antispam features of DA. I think Spamblocker/Exim does a good job when properly configured and uses little resources, but you can easily add extra layers of filtering if you feel the need to.
You should replace SpamAssassin with dspam though as it's more efficient and gives more power to the users, straight from the webmail.
 
Last edited:
interfasys - Thank you for the Pigeonhole tip. That looks like it should do exactly what I need. I do need to spend some time replacing SA with DSpam. I used to be involved with the project back before JZ sold it.

John - I have sa-update in crontab to run weekly and do have the latest version installed. I've owned my domain since 1994 and have had the same three letter email address since then (my initials). With SpamBlocker 4, I get around 75 spam messages per day (I'm sure this would be much higher without it, I have never bothered to count). SA will catch around 72% with a rare positive here or there when using 5 to flag spam.
 
Thanks. I will contact you, but maybe not before in a few weeks time, because I just have to much work to be done. In the meantime I have activated SpamAssassin in exim.conf, so I think I can wait a few weeks ...
 
Back
Top