Spamassassin user_prefs problems

[_rik_]

I'm writing here because my hosting provider cannot find the problem.
blacklist_from works OK.
Instead the rules I set in
#SAFE AREA start
#SAFE AREA end
are completely ignored.

e.g., I've tested

header EMPTY_ENVELOPE_FROM EnvelopeFrom =~ /^<>$/
describe EMPTY_ENVELOPE_FROM Empty envelope sender
score EMPTY_ENVELOPE_FROM 2.5

header EMPTY_ENVELOPE_USER Received =~ /\(envelope-from <>?\)/
describe EMPTY_ENVELOPE_USER Empty envelope sender detected in Received
score EMPTY_ENVELOPE_USER 2.5

header EMPTY_RETURN_PATH Return-Path =~ /^<>$/
describe EMPTY_RETURN_PATH Empty Return-Path header
score EMPTY_RETURN_PATH 2.5

None of the above works


Same problem for the contrary. None of the below works.

header EMPTY_ENVELOPE_FROM EnvelopeFrom =~ /\@return\.idealista\.it\b/
describe EMPTY_ENVELOPE_FROM Empty envelope sender
score EMPTY_ENVELOPE_FROM 2.5

header RETURN_IDEALISTA Return-Path =~ /\@return\.idealista\.it\b/i
describe RETURN_IDEALISTA Return-Path from return.idealista.it
score RETURN_IDEALISTA -2.5

#header FROM_IDEALISTA From =~ /@idealista\.it/i
#describe FROM_IDEALISTA Message from Idealista
#score FROM_IDEALISTA -2.5

but at least for this I can use the whitelist (despite is not exacty the same since there is no negative score case by case like in these rules)

Any idea why none of the above works and what could I try?

 

[zEitEr]

Can not find the place? Where is that? I would expect custom rules to be added in the file /etc/mail/spamassassin/local.cf

Instead the rules I set in
#SAFE AREA start
#SAFE AREA end
are completely ignored.

 

[_rik_]

Hi zEitEr,
I'm not an admin, I have only reseller permissions and I manage the user_prefs in any user account.
When you go in E-mail Manager --> Spamassassin Setup and click "Manually edit config file"
it opens the file /.spamassassin/user_prefs (where / is the user root).
Into such file you can see the safe area that is the part of the file that is not touched by the GUI.

 

[zEitEr]

Hi,

It's working on my end. I tested:

#SAFE AREA start
score    FREEMAIL_FROM  0.5

#SAFE AREA end

I change other settings of SpamAssassin in the browser and click SAVE just in case to make sure the things are reloaded fine.

 

[_rik_]

Hi Alex,
I'm sure user_prefs is reloaded because when I add new "blacklist_from" I see they actually work (I upload then via FTP with a script because I have to update ~ 100 accounts). It's only the rules that are ignored.

My hosting company confirms. They did some test and they opened a ticked with your support but after some days they told me nobody was able to solve the problem. It sound more like "we don't want to spend to much time on this" because I don't think it's so impossible... I mean I can't manage spamassassin command line because I'm a user, but if I had root privileges I'll surely find the culprit with some tests.

So without rules at the moment I can only cope with zillions of blacklist_from and stop words in "Spam Filters". :-/

 

[zEitEr]

So without rules at the moment I can only cope with zillions of blacklist_from and stop words in "Spam Filters". :-/

I did not test loading new rules in the user's file. But I confirm user can redefine scores in the file, I tested this moment on my end.
If you provide instructions on what rules you want me to test - I will test it.

 

[_rik_]

For example, I've tried 3 different ways to increase the score of email with no return path, and none of them works so it cannot be a problem of what field is available to spamassassin because I've tried any available fields.

header EMPTY_ENVELOPE_FROM EnvelopeFrom =~ /^<>$/
describe EMPTY_ENVELOPE_FROM Empty envelope sender
score EMPTY_ENVELOPE_FROM 2.5

header EMPTY_ENVELOPE_USER Received =~ /\(envelope-from <>?\)/
describe EMPTY_ENVELOPE_USER Empty envelope sender detected in Received
score EMPTY_ENVELOPE_USER 2.5

header EMPTY_RETURN_PATH Return-Path =~ /^<>$/
describe EMPTY_RETURN_PATH Empty Return-Path header
score EMPTY_RETURN_PATH 2.5

Same problem trying to decrease score in 3 different ways

header   EMPTY_ENVELOPE_FROM   EnvelopeFrom =~ /\@return\.idealista\.it\b/
describe EMPTY_ENVELOPE_FROM   Empty envelope sender
score    EMPTY_ENVELOPE_FROM   2.5

header   RETURN_IDEALISTA   Return-Path =~ /\@return\.idealista\.it\b/i
describe RETURN_IDEALISTA   Return-Path from return.idealista.it
score    RETURN_IDEALISTA   -2.5

#header   FROM_IDEALISTA   From =~ /@idealista\.it/i
#describe FROM_IDEALISTA   Message from Idealista
#score    FROM_IDEALISTA   -2.5

(In this latter case I've then used whitelist_from *@return.idealista.it but it's not the same level of granularity).

 

[zEitEr]

If you need to change only scores, then you don't need to specify "header" and "describe" lines. Or do you try to add completely new rules?

 

[_rik_]

Well... that's a nice tips to try because in some cases probably I could only change the score looking at the rule name in the email header. But for this case, I surely need to create a rule because there isn't any predefined rule for an empty return path, or is it?

 

[zEitEr]

I can confirm user defined rules saved in /home/<username>/.spamassassin/user_prefs are not used by SpamAssassin. Tested with

body LOCAL_DEMONSTRATION_RULE   /\btest2025\b/i
score LOCAL_DEMONSTRATION_RULE 7
describe LOCAL_DEMONSTRATION_RULE       This is a simple test rule

they do effect if loaded in e.g. /etc/mail/spamassassin/local.cf

But if to believe this page https://cwiki.apache.org/confluence/display/SPAMASSASSIN/WritingRules the users rules should be loaded as well.

~/.spamassassin/user_prefs is best if you want to have a rule only run when
a particular user runs SA.

 

[zEitEr]

OK, it seems the following line from the docs clarifies everything:

~/.spamassassin/user_prefs is best if you want to have a rule only run when a particular user runs SA.

For the custom rules added in the file to take any effect, SpamAssassin should run under the specific user. But by default spamassassin runs under root. So custom rules in the file do not take any effect, but scores can still override.

I tested in CLI, connected as the user in SSH:

$ spamassassin -D < 1762950903.M193674P13794.server.domain.net,S=6142,W=6265:2,

and my custom rule was loaded:

Nov 12 19:41:03.494 [25651] dbg: markup: mime_encode_header: No, score=-94.6 required=5.0 tests=BAYES_00,DKIM_SIGNED,
Nov 12 19:41:03.494 [25651] dbg: markup: [...] \tDKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,
Nov 12 19:41:03.494 [25651] dbg: markup: [...] \tGB_FREEMAIL_DISPTO,HTML_MESSAGE,LOCAL_DEMONSTRATION_RULE,
Nov 12 19:41:03.494 [25651] dbg: markup: [...] \tT_SPF_HELO_TEMPERROR,T_SPF_TEMPERROR,USER_IN_WELCOMELIST,
Nov 12 19:41:03.494 [25651] dbg: markup: [...] \tUSER_IN_WHITELIST autolearn=no autolearn_force=no version=4.0.1

See the test rule is in the line: LOCAL_DEMONSTRATION_RULE

 

[_rik_]

Thanks Alex,
it's all clear now.
It would be nice to have an improvement in this regard because spam is becoming more and more complex to block. For example many spam emails are crafted with no specific words to block, with no return-path, no envelop-from, random from... so rules would be great to make smarter filters.

 

[zEitEr]

It would be nice to have an improvement in this regard because spam is becoming more and more complex to block.

You might consider moving to a self-hosted solution, instead of a shared hosting then

 

[_rik_]

Of course that would solve the problem... but imagine this scenario: you have a car, let's say a Vauxall Corsa, and the FM stereo allows you to only receive from 80 to 85Mhz. You ask the seller how you can listen to Radio 105 and he says: "to extend the range you'd better buy a Porche Cayenne".

And that example is not even close because shared hosting is not only there because it's cheaper, it's there because managing a VPS is a lot of unnecessary extra work to justify the benefit of adding rules to spamassassin.

At that point it's easier/cheaper than a VPS to simply change panel, since the main DA competitor allows standard users to add new rules in the same user_prefs file (e.g., defining fresh header, body, meta, or other tests with their own describe and score), with those rules being honored and applied during scanning.

For this reasons, if I were the developer, I would think about it.

 

[zEitEr]

Of course that would solve the problem...

Actually the problem is clear. That's a limitation from SpamAssassin, not directadmin. And here I can see several solutions:

1. change hosting to another company, or use VPS
2. use a cheap VPS for filtering incoming emails
3. post a feature request, probably DirectAdmin developers will implement a ix for it

If you need any assistance from an independent administrator (it is me), you can PM me here or through my web-site.

 

[_rik_]

That's a limitation from SpamAssassin, not directadmin.

Yes and no... I mean: yes is spamassassin, so even changing hosting solves nothing. The problem is that spamassassin only allows "preferences" not rules in user_prefs.
However cPanel manage to overcome such limitation merging at scan-time. It has a mechanisms that allow user‑level configs to be merged into the active ruleset. In practice, this means if a user defines a header or body rule in their account’s config, cPanel ensures SpamAssassin actually evaluates it.
They achieve this by wrapping SpamAssassin with their own scripts and mail filters (Exim integration) that explicitly include user configs.
If they manage to do it, DA can do it as well... so I'll go with your point 3

Thanks again!