Inbound SpamAssassin scores high on ONE domain

[IT_Architect]

The inbound SpamAssassin scores high on ONE domain

We created a new domain about a week ago. All of the E-mails that are sent to that domain get scored high by SpamAssassin. All of the domains in this example are on the same server. One domain sends a message with two recipients. The message arrives at the new domain with a score of 4.4, and the other domain at 3.8. There are differences in the X-Spam-Status line.

X-Spam-Status to the 4.4 Account from the common sender:
X-Spam-Status: No, score=4.4 required=5.0 tests=ALL_TRUSTED,DNS_FROM_OPENWHOIS,FH_DATE_PAST_20XX,HTML_MESSAGE autolearn=no version=3.2.5

X-Spam-Status to the 3.8 Account from the common sender:
X-Spam-Status: No, score=3.8 required=5.0 tests=AWL,DNS_FROM_OPENWHOIS,FH_DATE_PAST_20XX autolearn=no version=3.2.5

What is also interesting is the difference in the X-Spam-Status line when both domains respond with an identical message to their common sender.

Reply From: X-Spam-Status on the 4.4 Account to the common sender:
X-Spam-Status: No, score=-0.1 required=5.0 tests=ALL_TRUSTED,BAYES_00,DNS_FROM_OPENWHOIS,FH_DATE_PAST_20XX,HTML_MESSAGE

Reply From: X-Spam-Status on the 3.8 Account to the common sender:
X-Spam-Status: No, score=0.2 required=5.0 tests=AWL,BAYES_00,DNS_FROM_OPENWHOIS,FH_DATE_PAST_20XX autolearn=no version=3.2.5

These are middle-of-the-road results based on a business e-mail using html stationary. With a text file the differences are much more pronounced. It such as 4.4 vs .6 and with more html, a score that makes somewhere in the 3s on one will reach 11 on the other.

SpamAssassin is basically unusable on the new user. Any thoughts as to why the differences?

 

[nobaloney]

The tests you point to are the tests that triggered the score. Have you looked at those tests? The extra tests triggered by the new user may be based on their domain name.

Jeff

 

[IT_Architect]

The tests you point to are the tests that triggered the score. Have you looked at those tests?

How can I look at tests of something that is done automatically?

The extra tests triggered by the new user may be based on their domain name.Jeff

I don't understand what you mean there. The domain has been parked for several years on one of our pages without showing links to other sites. In any case, the e-mails are scored high arriving at the new domain from other domains. When the new domain sends to other domains, the scores are low. Maybe I don't understand what you mean.

Thanks

 

[nobaloney]

You've got the list of failed tests.

You can find those tests within the SpamAssassin files and see what they're checking.

Just because the domain name has been parked doesn't mean it can't trap a test. Until you know exactly what those tests are looking for you don't know why the email is tripping on them.

So what I would do if I had the problem was check the tests to figure out what in the email is tripping them.

Alternatively you can find them and give them a lower spam score, or even a zero spam score.

Or turn it off inside DirectAdmin for that domain user account.

SpamAssassin is a wonderful open-source product, but it doesn't fit everyone needs.

Jeff

 

[IT_Architect]

Hi Jeff,

Just checking for an obvious answer. I'll track down why SpamAssassin on the new domain evaluates inbound e-mail from other domains with a much higher score than other domains receiving e-mail from the same domains, even on the same server. Fortunately the new domain doesn't have a bad name because when it sends e-mails, they are scored very low by the receiving domain. My only guess at this point before I check into it further is that there hasn't been enough ham and spam traffic yet for it to learn, but we'll see.

Thanks!

 

[tkret]

I confirm.
Domains, which have so far achieved score at 3-4 of about 2 weeks are 7-8.
At this time, users must set the High Threshold (10.0) to receive messages correctly. Previously, everything worked perfectly for Low Threshold (5.0)

What is the problem?

Sincerely,
Thomas

 

[IT_Architect]

I confirm.
Domains, which have so far achieved score at 3-4 of about 2 weeks are 7-8...What is the problem?<

When I analyzed the situation, I realized that the problem is exactly as Jeff stated, the rules. Unfortunately, the nature of the problem at this point is systemic. When reflected on what it would take to maintain the efficacy of SpamAssassin, I realized there are several deficiencies in the related areas that would need to be fixed in order to maintain consistent integrity of e-mail. Thus, I made a feature request that explains the issues and proposes a solution here:
http://directadmin.com/forum/showthread.php?t=35779
If you agree with the issues and proposed solution, that would be where to post your support for such a change.

Thanks!

 

[nobaloney]

I saw your new thread, but I still haven't seen specific things you've found. What specificallly needs to be done? Are you requesting a rewrite of certain rules? Here is not the right place for that. The right place for that is any support group maintained by the folk who maintain SpamAssassin. Otherwise we end up with a fork of the project, which we're responsible for maintaining.

Please note I have no issue with automatic installation of SpamAssassin and ClamAV; it would make life easier for me.

I just don't see how it solves the issue you've brought up here. Please expand.

Thanks.

Jeff

 

[IT_Architect]

I still haven't seen specific things you've found

I didn't, my partner found that open-whois.org no longer exists for the services that it used to provide. The problem was discovered and corrected last year. http://groups.google.com/group/linux.debian.bugs.dist/browse_thread/thread/490c97eb62641887. There may have been more of these instances since then. From reading, I see that it must be maintained to remain effective. For a new domain, this is a catastrophe because it has no Bayesian data to pull the score down. For seasoned domains, it means your threshold no longer works and you start missing E-mails. If you adjust it up, and then update the rules, now it's too loose. For our customers, their E-mail experience has more to do with how satisfied they are than anything else because it the most critical to them.

What specificallly needs to be done?

For installs: On the install side, the vast majority will want E-mail. If they have E-mail they will need Anti-Virus and Anti-Spam, so it should be part of the install. If it is part of the install, there needs to be a way to maintain it, such as sa-update, which means what is necessary for that needs to be part of the install. That could probably be chroned. Server admins won't maintain their Anti-spam by hand anymore than they will their Anti-virus.

I just don't see how it solves the issue you've brought up here. Please expand.

For Updates: An important part of keeping our servers secure and running well is through updates. DirectAdmin has provided us with a method to know every day what needs to be updated. The problem is there are penalties for doing so. Updates overwrite exim.conf about 2/3rds of the time. Thus, after doing an update to maintain the safety and integrity of the server, we are left with no Anti-Virus, no Anti-Span, and for me, I lost my setting for users to get their large E-mails. Then they call me up with the message size issue, and/or talk about a flood of spam coming in. Admins don't want to save off an old copy of the exim.conf, and overwrite the new one either nor do they want to pick through the file and re-modify it each time the apply updates and restart services. Updates won't happen in a timely manner under these circumstances. What I'm proposing for updates, removes the penalties for keeping our servers up to date.

 

[nobaloney]

I didn't, my partner found that open-whois.org no longer exists for the services that it used to provide. The problem was discovered and corrected last year. http://groups.google.com/group/linux.debian.bugs.dist/browse_thread/thread/490c97eb62641887.

And hopefully you now know that you need to keep your anti-spam and anti-virus solutions updated.

There may have been more of these instances since then. From reading, I see that it must be maintained to remain effective.

Of course it does. Everything on your server needs to be maintained to be effective.

For our customers, their E-mail experience has more to do with how satisfied they are than anything else because it the most critical to them.

My customers feel the same way. That's why I maintain my anti-spam and my anti-virus solutions more often than anything else except fixes for known exploits in the wild.

Server admins won't maintain their Anti-spam by hand anymore than they will their Anti-virus.

Then they're making a big error in their server administration. As an experienced administrator of lots of servers I understand that there's no one anti-spam or anti-virus setup that works for everyone, for lots of reasons. I expect to have to maintain it myself.

The problem is there are penalties for doing so. Updates overwrite exim.conf about 2/3rds of the time.

That should never happen. Please post the specific kind of updates that overwrit exim.conf so we (those of us who work on these things) can investigate your problem.

Thus, after doing an update to maintain the safety and integrity of the server, we are left with no Anti-Virus, no Anti-Span, and for me, I lost my setting for users to get their large E-mails.

Again, this should not happen. It's never happened for me, and I do automatic updates and semi-automatic updates, utilizing automatic nightly running of yum, and running CustomBuild as required. but do note that as required. You shouldn't be fixing something that isn't broken. We monitor CustomBuild's emails, but we don't update just because there's an update available.

Once you begin updating only as required, you'll be able to take the time to check, so even if something untoward is happening to your exim.conf file, you'll be able to schedule what you do, and when you do it, and check to make sure you don't have a problem.

And of course there's nothing to stop you from making copies of important files. I make a copy of exim.conf every time I change it, or run something that might change it, and then I do a diff afterwards. That's how I know it's not changing. But if it ever did, I'd simply fix it.

Then they call me up with the message size issue, and/or talk about a flood of spam coming in. Admins don't want to save off an old copy of the exim.conf, and overwrite the new one either nor do they want to pick through the file and re-modify it each time the apply updates and restart services.

I don't think that's true at all for committed systems administrators. I think we take pride in what we do, and our value to the companies we work for. I know that I do. Your exim.conf file should not be overwritten, but there's no reason why you can't just copy over your last one if it does.

And again, if you post under what circumstances exim.conf file gets overwritten, that's fixable; it shouldn't be happening if you're using updates supplied as standard RPMs, DEBs, or DirectAdmin files.

Updates won't happen in a timely manner under these circumstances.

Updates shouldn't always happen in a timely manner. However I admit it may take some level of experience to understand when updates and when they're not, important to do promptly.

What I'm proposing for updates, removes the penalties for keeping our servers up to date.

DirectAdmin definitely requires more self-administration than some other control panels; that's because it's more flexible. It presumes decisions will be made by systems administrators more than do some other control panels.

Jeff

 

[IT_Architect]

And hopefully you now know that you need to keep your anti-spam and anti-virus solutions updated.

Oh no you don't. Anti-virus and anti-spam are not things people maintain by hand. In this case the anti-spam updating is broken because it isn't even installed. There aren't a thousand different ways the people handle spam in DirectAdmin. There is basically just one, SpamAssassin.

I explained the exim.conf fiasco when there is an update. This has been going on for a long time. That part is a mess.

 

[nobaloney]

Oh no you don't.

In my opinion that's an incredibly dangerous attitude to take. If you take it your clients will complain they get too much spam, and eventually your servers will get blacklisted because they send spam.

But you probably know that, and that's probably why you want DirectAdmin staff to do your work for you.

The problem as I've stated previously is that different servers and different admins require different solutions.

Okay, let's then agree to disagree, since I explained that I've never seen exim.conf get overwritten and you don't seem to want to explain the specific things you did that caused it to be overwritten.

However, here's the problem as I see it:

DirectAdmin staff chooses to give you commandline tools to install both SpamAssassin and ClamAV on your server, but to not do it for you.

Guess what? I'd love it if they did, because then it would be easier for me to write the exim.conf file included in DirectAdmin.

Yes, I'm the guy who writes the exim.conf file which manages SpamAssassin and ClamAV on DirectAdmin servers.

You don't like how it works. Doesn't that leave you with writing your own solution?

Now don't get me wrong; I know you're going to want to write back that DirectAdmin staff should do it. You've written that a few times already. Your point has been made.

They've written previously that they don't always see all posts made to these forums. Have you written them to make sure they look at this one?

Jeff