Low ESF score

harro

Verified User
Joined
Oct 15, 2005
Messages
178
** please note, this is NOT just about a low ESF score - this is about ESF NOT WORKING **


Good day all,

After following the instructions to install Easy Spamfighter (and the Spamblocker) via Custombuild, I find that spam is scored, but no action is taken bij Exim when the spam is (obviously) spam:

Message-Id: <[email protected]>
Date: Tue, 24 May 2016 05:57:00 +0700
SPFCheck: Soft Fail, 30 Spam score
BlacklistCheck: Blacklisted address, +50 Spam score
SpamTally: Final spam score: 80

Does anyone else have this same problem and what could be done to trigger Exim to actually block or delete non-accepted (high scoring) unwanted emails?

Thank you and kind regards,
Harro
 
Last edited:
Hello,

I've moved your post to a new thread under the ESF forum.

I'm not seeing any "X-Spam-Score" header there, so SpamAssassin isn't adding it's contribution to the score.
Make sure that SpamAssassin is enabled in the options.conf and spamd is running, then do "./build exim_conf" again.

http://help.directadmin.com/item.php?id=576

John
 
Hi John,

Excuse me for only quoting the Spamblocker output, please see the full output below (including spamassassin):


Date: Tue, 24 May 2016 05:53:18 +0700
BlacklistCheck: Blacklisted address, +50 Spam score
X-Spam-Score: 1.3 (+)
X-Spam-Report: Spam detection software, running on the system "servername",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.

Content preview: Hi name, Please see the report attached I mentioned
in my last email Thank you, Maureen Lloyd Cogent Communications Holdings,
Inc. [...]

Content analysis details: (1.3 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
1.3 RDNS_NONE Delivered to internal network by a host with no rDNS
SpamTally: Final spam score: 63


So, Spamassassin gives it 1.3 points and Spamblocker seems to convert those 1.3 points to 13 (+50 points for blacklisted sender IP).


The output from a ps aux on the server(s):


]# ps aux | grep -i spam
root 723 0.0 0.2 174116 4876 ? Ss Apr14 9:58 /usr/bin/spamd --pidfile /var/run/spamd.pid -d -c -m 15 --ipv4
root 5708 0.0 2.8 185248 59304 ? S 00:57 0:12 spamd child
root 23598 0.0 2.1 175736 46044 ? S May21 0:03 spamd child
root 31917 0.0 0.0 8988 936 pts/1 S+ 19:26 0:00 grep --color=auto -i spam

There is no change in the subject line and the email is delivered as a normal email would be. i.e. it seems that only the Spamassassin score is taken into account, ignoring the Spamblocker score.

This problem occurs on two servers (I only updated two servers thus far).

Kind regards,

Harro


p.s. a second example (I deleted most of the other spam):

X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on servername
X-Spam-Level: *
X-Spam-Status: No, score=1.3 required=5.0 tests=RDNS_NONE autolearn=no
autolearn_force=no version=3.4.1
Received: from [95.107.173.210] (helo=unknown.tring.al)
by servername with esmtp (Exim 4.86)
(envelope-from <[email protected]>)
id 1b3nPD-0002ag-03
for my-email-address; Fri, 20 May 2016 18:36:12 +0200
Received: by mail.serverdomain.local (Postfix, from userid 773)
id 0815901408; Fri, 20 May 2016 18:36:00 +0200
To: my-email-address
Subject: Re:
From: "Alice Clarke" <[email protected]>
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="------------25a695945f475c9be775af82b73f6b3f"
Message-Id: <[email protected]>
Date: Fri, 20 May 2016 18:36:00 +0200
SPFCheck: Soft Fail, 30 Spam score
BlacklistCheck: Blacklisted address, +50 Spam score
SpamTally: Final spam score: 80
 
Last edited:
Hello,

I've moved your post to a new thread under the ESF forum.

( ... )

John


Good day John,

On a side note: while I agree that this question is good in the ESF forum, I am not sure that the topic "Low ESF score" really reflects the issue (I have seen scores of 60-93 from ESF, which seems high to me, unless the threshold is 100 or more).

It seems more a problem of ESF score not triggering action within Spamassassin / ESF not working.

Kind regards,

Harro
 
Reading the headers above, a SpamAssassin score of 1.3 doesn't qualify as spam at all, henec the header:
Code:
[COLOR=#333333]X-Spam-Status: No, score=1.3 required=5.0 tests=RDNS_NONE autolearn=no[/COLOR]
and ESF multiplies 1.3 by 10, giving 13.
That's added to the BlackList check score of 50:
Code:
[COLOR=#333333][I]BlacklistCheck: Blacklisted address, +50 Spam score[/I][/COLOR]
which is so 50+13 = 63, hence that's the final spam score.

In any case, there are not enough points to justify it as being spam, hence the subject isn't being changed.

One thing though.. SpamAssassin is finding RDNS_NONE... but ESF is not finding that.. via the EASY_NO_REVERSE_IP score.
The EASY_NO_REVERSE_IP score alone should be 100... so 163, but it's not getting caught.
We'd need to find out why.... so check:
Code:
/var/log/exim/mainlog*
for all entries relating to message ID 1b3nPD-0002ag-03...
In and around those entries, ESF may be logging things, like if the sender IP is in a whitelist or something like that.

The ESF check for rDNS is done in:
Code:
/etc/exim.easy_spam_fighter/check_mail.conf
so any "accept" that happens before the bottom of the file (where the rdns check lives) would cause that check to be skipped.

Looking at the file, the only cases that could have it accepted early would be:
- sender IP in the relay_hosts, but you'd see a "PrimaryMX" header added.
- authenticated, which is possible. The exim mainlog would show us if they've authenticated to deliver you the message.
- sender is skipped via esf_skip_senders, in which case you'd see "[email protected] skipped via esf_skip_senders" in the exim mainlog

and there are also more possible accept conditions in the /etc/exim.conf "acl_check_mail" before the ESF check_mail.conf is even called:
- whitelist_domains (ensure /etc/virtual/whitelist_domains is empty)
- whitelist_hosts
- whitelist_hosts_ip
- whitelist_senders
all of which should be logging something to the exim mainlog.

John
 
Hi John,

Thank you for the elaborate response. What I find in the mainlog for the email is:

]# fgrep 1b3nPD-0002ag-03 mainlog-20160523
2016-05-20 18:36:12 1b3nPD-0002ag-03 <= [email protected] H=(unknown.tring.al) [95.107.173.210] P=esmtp S=3193 [email protected] T="Re:" from <[email protected]> for <myemailaddress>
2016-05-20 18:36:12 cwd=/var/spool/exim 3 args: /usr/sbin/exim -Mc 1b3nPD-0002ag-03
2016-05-20 18:36:17 1b3nPD-0002ag-03 => <myemailaddress> F=<[email protected]> R=spamcheck_director T=spamcheck S=3454
2016-05-20 18:36:17 1b3nPD-0002ag-03 Completed


To me this does not trigger an 'ah ha!' as to why the Easy Spam fighter is not functioning fully. As mentioned, I have installed the ESF on two servers following the Custombuild instructions in the original post where I posted my message above. On both servers these emails are not being blocked or marked as spam.

As a practical option, can I lower the ESF threshold for blocking/renaming an email? What is currently the limit for ESF to go into action?

Thank you and kind regards,

Harro
 
There may be more relevant info that does not contain the message ID.. try:
Code:
less mainlog-20160523
and press / to search for the message ID. There might be entries in between which as more info.

Yes, you can change the limit.
Add the variable to
Code:
/etc/exim.easy_spam_fighter/variables.conf.custom
to override the original.
You might need to use == instead of = to override them.

John
 
Good day John,

Excuxe me for the slow response, I am traveling. Looking in the logs *around* the lines that I quoted in my previous message, I note that ESF logs a penalty of +100 Spam score for No rDNS, however this does not get used / shown in the email header.

So the question is whether ESF/Spamassassin is not reading the penalty for No rDNS, or it is reading is (and ignoring it) but not reporting in the email?


2016-05-20 18:36:12 1b3nPD-0002ag-03 <= [email protected] H=(unknown.tring.al) [95.107.173.210] P=esmtp S=3193 [email protected] T="Re:" from <[email protected]> for myemailadress
2016-05-20 18:36:12 cwd=/var/spool/exim 3 args: /usr/sbin/exim -Mc 1b3nPD-0002ag-03
2016-05-20 18:36:12 cwd=/tmp 4 args: /usr/sbin/exim -oMr spam-scanned -bS
2016-05-20 18:36:14 ReverseDNS: No reverse DNS for mailserver at 117.3.70.73, +100 Spam score
2016-05-20 18:36:17 1b3nPI-0002an-MQ <= [email protected] U=mail P=spam-scanned S=3572 [email protected] T="Re:" from <[email protected]> for myemailadress
2016-05-20 18:36:17 cwd=/var/spool/exim 3 args: /usr/sbin/exim -Mc 1b3nPI-0002an-MQ
2016-05-20 18:36:17 1b3nPI-0002an-MQ => <myemailadress> F=<[email protected]> R=virtual_user T=dovecot_lmtp_udp S=3687 C="250 2.0.0 <[email protected]> +maONAE9P1fHIwAA20gPiA Saved"
2016-05-20 18:36:17 1b3nPI-0002an-MQ Completed
2016-05-20 18:36:17 1b3nPD-0002ag-03 => <myemailadress> F=<[email protected]> R=spamcheck_director T=spamcheck S=3454
2016-05-20 18:36:17 1b3nPD-0002ag-03 Completed


I hope I am not confusing matters, but please find below the header from another spam email that I received yesterday (score 98), and what I found in the logs (basically nothing, despite the high 98 score):

Delivery-date: Fri, 27 May 2016 22:33:50 +0200
Received: from [188.86.131.168] (helo=static-168-131-86-188.ipcom.comunitel.net)
by servername with esmtp (Exim 4.87)
(envelope-from <[email protected]>)
id 1b6ORw-0004Aw-Li
for myemailaddress; Fri, 27 May 2016 22:33:50 +0200
Date: Fri, 27 May 2016 22:33:28 +0200
Message-Id: <201605272233.gfljDmlZvHzAhf@myemailaddress>
To: myemailaddress
Subject: New Invoice
X-PHP-Script: travelblissful.com/mail/send_script.php for 98.160.129.169, 98.160.129.169
MIME-Version: 1.0;
Content-Type: multipart/mixed; boundary="--c90a2446a00797d099698f3ae473dfca"
From: "Marquita Forbes" <[email protected]>
X-SA-Exim-Connect-IP: 10.10.19.38
X-SA-Exim-Mail-From: travelblissful.com
X-SA-Exim-Scanned: No (on travelblissful.com); SAEximRunCond expanded to false
BlacklistCheck: Blacklisted address, +50 Spam score
X-Spam-Score: 4.8 (++++)
X-Spam-Report: Spam detection software, running on the system "appa.pakanet.nl",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.

Content preview: Dear info, I appreciate your speaking with me today. Per our
conversation, please find attached invoice. Please do not hesitate to contact
me with any questions you may have. Thank you for your time and consideration.
[...]

Content analysis details: (4.8 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
0.2 CK_HELO_GENERIC Relay used name indicative of a Dynamic Pool or
Generic rPTR
1.3 RDNS_NONE Delivered to internal network by a host with no rDNS
3.2 HELO_DYNAMIC_IPADDR Relay HELO'd using suspicious hostname (IP addr
1)
SpamTally: Final spam score: 98

Aside from the fact that the score noted in the email adds up to 97 (where did the extra ooint come from), there is no mention of any spam evaluation in the mainlog:

2016-05-27 22:33:50 1b6ORw-0004Aw-Li <= [email protected] H=(static-168-131-86-188.ipcom.comunitel.net) [188.86.131.168] P=esmtp S=11316 id=201605272233.gfljDmlZvHzAhf@servername T="New Invoice" from <[email protected]> for myemailaddress
2016-05-27 22:33:50 cwd=/var/spool/exim 3 args: /usr/sbin/exim -Mc 1b6ORw-0004Aw-Li
2016-05-27 22:33:51 1b6ORw-0004Aw-Li => info <myemailaddress> F=<[email protected]> R=virtual_user T=dovecot_lmtp_udp S=11623 C="250 2.0.0 <myemailaddress> xiuTOC6vSFd/OQAA20gPiA Saved"
2016-05-27 22:33:51 1b6ORw-0004Aw-Li Completed

Since there do not seem to be other users facing the same problem (no reactions, no other threads about this topic, it would suggest that it is a local problem. However, being the same problem on at least two servers it could also suggest a particular (identical) upgrade path that leaves out some config update that triggers the proble
 
Last edited:
Hi,

I experience the same problem actually. Below are some of the logs on my machine:


The first:
Return-Path: <[email protected]>
Delivered-To: mymailaddress
Received: from server03.dannik.eu
by server03.dannik.eu (Dovecot) with LMTP id g5eGFOtNTFdPOgAAkS2Xqg
for <mymailaddress>; Mon, 30 May 2016 16:27:55 +0200
Return-path: <[email protected]>
Envelope-to: <mymailaddress>
Delivery-date: Mon, 30 May 2016 16:27:55 +0200
Received: from mail by server03.dannik.eu with spam-scanned (Exim 4.86.2)
(envelope-from <[email protected]>)
id 1b7OAc-0003sa-My
for <mymailaddress>; Mon, 30 May 2016 16:27:55 +0200
X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on server03.dannik.eu
X-Spam-Level: **
X-Spam-Status: No, score=2.3 required=4.6 tests=BAYES_50,HTML_EXTRA_CLOSE,
HTML_MESSAGE,LOTS_OF_MONEY,RCVD_IN_BRBL_LASTEXT autolearn=no
autolearn_force=no version=3.4.1
Received: from [190.104.112.242]
by server03.dannik.eu with esmtp (Exim 4.86.2)
(envelope-from <[email protected]>)
id 1b7OAX-0003sV-Dq
for mymailaddress; Mon, 30 May 2016 16:27:54 +0200
Date: Mon, 30 May 2016 08:27:37 -0600
Subject: Community Fine
Message-ID: <[email protected]>
From: Ginnifer Middleton <[email protected]>
To: <mymailaddress>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="--_com.boxer.email_5627466452425"
ReverseDNS: No reverse DNS for mailserver at 190.104.112.242, +100 Spam score
BlacklistCheck: Blacklisted address, +50 Spam score
Subject: ****SPAM***** _SCORE_/_REQD_ Community Fine
SpamTally: Final spam score: 170
X-Antivirus-Scanner: Clean mail though you should still use an Antivirus

2016-05-30 16:27:54 1b7OAX-0003sV-Dq ESF score is 56 / 300
2016-05-30 16:27:54 1b7OAX-0003sV-Dq <= [email protected] H=([190.104.112.242]) [190.104.112.242] P=esmtp S=10188 [email protected] T="Community Fine\n*****SPAM***** _SCORE_/_REQD_ Community Fine" from <[email protected]> for <mymailaddress>
2016-05-30 16:27:54 cwd=/ 3 args: /usr/sbin/exim -Mc 1b7OAX-0003sV-Dq
2016-05-30 16:27:54 cwd=/ 4 args: /usr/sbin/exim -oMr spam-scanned -bS
2016-05-30 16:27:55 1b7OAc-0003sa-My <= [email protected] U=mail P=spam-scanned S=10615 [email protected] T="Community Fine\n*****SPAM***** _SCORE_/_REQD_ Community Fine" from <[email protected]> for <mymailaddress>
2016-05-30 16:27:55 cwd=/ 3 args: /usr/sbin/exim -Mc 1b7OAc-0003sa-My
2016-05-30 16:27:55 1b7OAc-0003sa-My => <mymailaddress> F=<[email protected]> R=virtual_user T=dovecot_lmtp_udp S=10924 C="250 2.0.0 <mymailaddress> g5eGFOtNTFdPOgAAkS2Xqg Saved"
2016-05-30 16:27:55 1b7OAc-0003sa-My Completed
2016-05-30 16:27:55 1b7OAX-0003sV-Dq => <my2ndmailaddress> F=<[email protected]> R=spamcheck_director T=spamcheck S=10492
2016-05-30 16:27:55 1b7OAX-0003sV-Dq Completed


And the second:
Return-Path: <[email protected]>
Delivered-To: <mymailaddress>
Received: from server03.dannik.eu
by server03.dannik.eu (Dovecot) with LMTP id 5SWKBc0ZTFddKQAAkS2Xqg
for <mymailaddress>; Mon, 30 May 2016 12:45:33 +0200
Return-path: <[email protected]>
Envelope-to: <myothermailadress>
Delivery-date: Mon, 30 May 2016 12:45:33 +0200
Received: from [182.68.200.148] (helo=abts-north-dynamic-148.200.68.182.airtelbroadband.in)
by server03.dannik.eu with esmtp (Exim 4.86.2)
(envelope-from <[email protected]>)
id 1b7KhP-0002kf-FH
for <myothermailadress>; Mon, 30 May 2016 12:45:32 +0200
Date: Mon, 30 May 2016 16:15:18 +0530
Subject: Community Fine
Message-ID: <[email protected]>
From: Keontrey Gonzalez <[email protected]>
To: me <myothermailadress>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="--_com.boxer.email_4836463946347"
BlacklistCheck: Blacklisted address, +50 Spam score
X-Spam-Score: 3.3 (+++)
X-Spam-Report: Spam detection software, running on the system "server03.dannik.eu",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.

Content preview: You have been charged with the violation of community rules
[AR.395/92] and obliged to pay a fine amounting to $5,506,83. Please refer
to the document enclosed for more information. Community Fine You have been
charged with the violation of community rules [AR.395/92] and obliged to
pay a fine amounting to $5,506,83. Please refer to the document enclosed for
more information. [...]

Content analysis details: (3.3 points, 4.6 required)

pts rule name description
---- ---------------------- --------------------------------------------------
0.2 CK_HELO_GENERIC Relay used name indicative of a Dynamic Pool or
Generic rPTR
0.0 HTML_EXTRA_CLOSE BODY: HTML contains far too many close tags
0.0 HTML_MESSAGE BODY: HTML included in message
0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60%
[score: 0.4988]
1.4 RCVD_IN_BRBL_LASTEXT RBL: No description available.
[182.68.200.148 listed in bb.barracudacentral.org]
0.0 LOTS_OF_MONEY Huge... sums of money
0.8 RDNS_NONE Delivered to internal network by a host with no rDNS
SpamTally: Final spam score: 83
X-Antivirus-Scanner: Clean mail though you should still use an Antivirus

2016-05-30 12:45:32 1b7KhP-0002kf-FH <= [email protected] H=(abts-north-dynamic-148.200.68.182.airtelbroadband.in) [182.68.200.148] P=esmtp S=11870 [email protected] T="Community Fine" from <[email protected]> for <mymailaddress>
2016-05-30 12:45:32 cwd=/ 3 args: /usr/sbin/exim -Mc 1b7KhP-0002kf-FH
2016-05-30 12:45:33 1b7KhP-0002kf-FH => <mymailaddress> F=<[email protected]> R=virtual_user T=dovecot_lmtp_udp S=12195 C="250 2.0.0 <mymailaddress> 5SWKBc0ZTFddKQAAkS2Xqg Saved"
2016-05-30 12:45:33 1b7KhP-0002kf-FH [176.56.227.99] SSL verify error: depth=0 error=self signed certificate cert=/C=GB/ST=Someprovince/L=Sometown/O=none/OU=none/CN=localhost/emailAddress=webaster@localhost
2016-05-30 12:45:33 1b7KhP-0002kf-FH [176.56.227.99] SSL verify error: certificate name mismatch: "/C=GB/ST=Someprovince/L=Sometown/O=none/OU=none/CN=localhost/emailAddress=webaster@localhost"
2016-05-30 12:45:33 1b7KhP-0002kf-FH => <mymailaddress> F=<[email protected]> R=lookuphost T=remote_smtp S=12059 H=mail.dannik.nl [176.56.227.99] X=TLSv1.2:AES256-GCM-SHA384:256 CV=no C="250 OK id=1b7KhR-0004UU-5R"
2016-05-30 12:45:33 1b7KhP-0002kf-FH Completed

Just for logging I slightly altered the default settings for ESF (I want to check wether the spam gets detected, when it's deleted immediately I can't check):
# cat /etc/exim.easy_spam_fighter/variables.conf.custom
EASY_HIGH_SCORE_DROP == 180
EASY_SPAMASSASSIN_MAX_SIZE == 800K

I find it quite strange the first mail is marked as spam (the subject is altered), however I still receive the message in my inbox instead of the spambox. Also Spamassassin doesn't recognize it as being spam. Both mails were received on the same domain; different addresses, although both adressess are forwarded to the same mailbox (the second mail is even forwarded to 2 separate mailboxes on different domains).

I must say installation of ESF has dropped the amount of spam a bit, unfortunately still quite a lot of spam like the above comes through.

Regards,
Danny
 
Hi John, Dannik,

Although two is not a crowd, there are now two independent users (and at least three independent servers) where there is an issue with Easy Spam Fighter not functioning as it should.

Are there any other admins with such an issue? Perhaps the problem goes by unnoticed since there are no errors, just more spam being allowed through than should be?

Kind regards,

Harro

p.s. I would be in favour of changing the thread title to something more suitable such as "ESF score not functioning" or "problem with link ESF - Spamassassin" to sollicit more responses. "Low ESF score" just seems like a luxury problem that may not trigger any recognition...
 
Last edited:
Same thing is happening on our servers. ESF gives the email 70 points, which is above the 55 treshold, but the email still gets through.
 
If my memory is correct, there are 2 ways to drop a message with ESF.

1) It will respect the SA settings, so you'd have to actually set the high scoring email to be dropped if you want it dropped.

It's all controlled in:
/etc/exim.easy_spam_fighter/check_message.conf

with various called to the /etc/exim.pl, which looks in /etc/virtual/domain.com/filter.conf

2) But the actual ESF high score block should work, but there are a few conditions that must be true:
Code:
    condition = ${if !eq{$acl_m_esf_skip}{1}}
    condition = ${if >={$acl_m_easy69}{EASY_HIGH_SCORE_DROP}{1}{0}}
    condition = ${if !eq{$acl_m_spam_user}{nobody}}
where it's most likely that the acl_m_esf_skip is already set to 1 from some other area.

Check:
/var/log/exim/mainlog

to see if it mentions skipping the incoming message checks anywhere..
Possible causes are any of the *skip* or *whitelist* files in:
/etc/virtual/*

but could also be the whitelistrbl, which we've recently changed to only allow medium-high in 4.4.5+
Again, should be logged in /var/log/exim/mainlog

John
 
Back
Top