SpamAssassin Version not showing in header

IT_Architect

Verified User
Joined
Feb 27, 2006
Messages
888
FreeBSD 11.2

I only have one of these newer servers in active use with DA that uses CB2 and up-to-date version of SpamAssassin. With the older CB, I edited the text files by hand and never had an issue. It looks like it sort of does get scanned by SpamAssassin, but what I'm used to seeing with the old isn't there. (The X-Text-Classification at the end is from POPFile running on my laptop.) I can go back through SpamBlocker install and extra modules but I'm not sure what headers should look like after blockcracking, + easy_spam_fighter, + SpamAssassin, nor how to determine if everything is working correctly, nor whether I can do a rewrite configs without breaking things if I don't edit the configs after install. If this is the new header format, then I need to change the rules in Outlook to properly file spam. With the old headers, I just needed to read X-Spam-Status.

It appears to be running OK
Code:
root@server:~ # ps -ax | grep spamd
12340  -  Ss      0:04.16 /usr/local/bin/perl -T -w /usr/bin/spamd -d -c -m 15
12341  -  I       0:13.37 spamd child (perl)
12342  -  I       0:00.90 spamd child (perl)
16612  0  D+      0:00.00 grep spamd
The new headers look like this:
Code:
Return-Path: <John.Smith@SendingEnd.com>
Delivered-To: Jim.Jones@ReceivingEnd.com
Received: from server.ReceivingEnd.com
	by server.ReceivingEnd.com with LMTP
	id QLHxMmzR6lswLwAAnMTBGA
	(envelope-from <John.Smith@SendingEnd.com>)
	for <Jim.Jones@ReceivingEnd.com>; Tue, 13 Nov 2018 08:28:12 -0500
Return-path: <John.Smith@SendingEnd.com>
Envelope-to: Jim.Jones@ReceivingEnd.com
Delivery-date: Tue, 13 Nov 2018 08:28:12 -0500
Received: from server1.emailserver.com ([57.128.088.184])
	by server.ReceivingEnd.com with esmtps (TLSv1:AES256-SHA:256)
	(Exim 4.91)
	(envelope-from <John.Smith@SendingEnd.com>)
	id 1gMYjW-0003EN-RB
	for Jim.Jones@ReceivingEnd.com; Tue, 13 Nov 2018 08:28:12 -0500
Received: from c-68-56-95-133.hsd1.mi.comcast.net ([68.56.95.133] helo=M6800)
	by server1.emailserver.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256)
	(Exim 4.80.1)
	(envelope-from <John.Smith@SendingEnd.com>)
	id 1gMYjV-000DmW-Uz
	for Jim.Jones@ReceivingEnd.com; Tue, 13 Nov 2018 08:27:58 -0500
From: "John.Smith" <John.Smith@SendingEnd.com>
To: <Jim.Jones@ReceivingEnd.com>
Subject: Test SpamAssassin
Date: Tue, 13 Nov 2018 08:27:51 -0500
Organization: My Business, LLC
Message-ID: <006501d47b54$ae706da0$0b5148e0$@SendingEnd.com>
MIME-Version: 1.0
Content-Type: multipart/related;
	boundary="----=_NextPart_000_0066_01D47B2A.C59C1350"
X-Mailer: Microsoft Outlook 14.0
Thread-Index: AdR7VKbc/VOfwb6uSIqiw/eMBWJSXg==
Content-Language: en-us
Forward-Confirmed-ReverseDNS: Reverse and forward lookup success on 57.128.088.184, -10 Spam score
SPFCheck: Server passes SPF test, -30 Spam score
X-Spam-Score: 0.0 (/)
X-Spam-Report: Spam detection software, running on the system "server.ReceivingEnd.com",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  <http://www.SendingEnd.com/> My Business,
    LLC Hi, Test Message text goes here Sincerely yours, cid:image002.gif@01D47B26.19F50A10
    John.Smith My Business, LLC Phone (789)123-456 <http://www.SendingEnd.com>
    http://www.SendingEnd.com [...] 
 Content analysis details:   (0.0 points, 5.0 required)
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
  0.0 URIBL_BLOCKED          ADMINIS
TRATOR NOTICE: The query to URIBL was
                             blocked.  See
                             http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
                              for more information.
                             [URIs: SendingEnd.com]
 -0.0 SPF_PASS               SPF: sender matches SPF record
  0.0 HTML_IMAGE_RATIO_06    BODY: HTML has a low ratio of text to image
                             area
  0.0 HTML_MESSAGE           BODY: HTML included in message
SpamTally: Final spam score: -40
X-Antivirus-Scanner: Clean mail though you should still use an Antivirus
X-Text-Classification: other
The old headers look like this:
Code:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
	emailserver.com
X-Spam-Level: 
X-Spam-Status: No, score=-1.1 required=4.0 tests=BAYES_00,HTML_MESSAGE,
	RDNS_NONE,SPF_HELO_PASS,SPF_PASS,URIBL_BLOCKED autolearn=no version=3.3.1
X-Spam-Relay-Country: US US
 
Last edited:

IT_Architect

Verified User
Joined
Feb 27, 2006
Messages
888
I SOLVED my questions myself and decided to share what I learned here to save others some hair follicles trying to figure out if, what, and how their anti-spam efforts are working.

Problem: Due to unfamiliarity with the new (to me) improvements in spam fighting software, the normal tags that used to be incorporated into the email headers that indicated messages were being scanned were no longer there. Nobody answered my questions, so I researched how to determine what was going on, and decided to document that here for others who may have similar questions could find learn from this without having to read many threads and sites like I did.

Environment: FreeBSD 11.2 64bit server, DirectAdmin 1.54.1, with binaries and scripts last updated 11/11/2018, set up using the standard CB 2.0 install scripts as documented here:
SpamBlocker install and extra modules.

SpamBlocker is actually an Exim configuration that was developed by the late Jeff Lassman/AKA nobaloney of NoBaloney Internet Svcs. The file(s) have hooks and parameters that stitch together a system to defeat spam without us having to weave through all of the pitfalls of putting together a system like this. Today, DirectAdmin branched off of Jeff's SpamBlocker, continues to change and improve it to take advantage of new and changing software capabilities, and has integrated it with CB 2.0 scripts. This requires keeping exim.conf and exim.pl versions paired. You cannot update one without the other. The new exim.pl will not work for older exim.conf files. The new exim.pl will not work for older exim.conf files, nor will the new exim.conf work with old exim.pl files. The enhanced effectiveness comes at the expense of greater complexity, which makes it impractical to manage these files manually. However, unlike before, you can now rewrite configs without backing up any files or breaking anything as long as you follow the procedures outlined here SpamBlocker install and extra modules. In this case, I selected all of the options. What follows is a brief description of what they do:

A. SMTP-time ACL blocking improvement - When a USER, which includes the total of all of virtual E-Mail users, reaches a certain level of E-Mails sent, it will stop him from sending any more, but instead of returning a failed password at when the E-Mail limit is reached for the user, it will return the a "too many sends" message. Because this occurs at the SMTP OUTBOUND level, there will be no header entry.

B. blockcracking - BlockCracking is where exim keeps track of how many OUTBOUND non-existent emails a sender tries to send within a given period of time, and blocks the VIRTUAL E-MAIL USER from sending if the count is too high. The default way to unblock the virtual e-mail user's account is by the changing of their password, which the user can do in the control panel. Because this occurs at SMTP OUTBOUND, there will be no header entry from blockcracking.

C. easy_spam_fighter - EasySpamFigther for INBOUND SMTP-time spam scanning/scoring and blocking, including reverse IP lookups, DKIM/SPF checks, and smtp-time SpamAssassin runs based on the actual ~/.spamassasin/user_prefs. Because this occurs at the SMTP INBOUND level, there WILL be a header entry if it is operational. This occurs prior to processing by the more resource intensive spamassassin. If the score already exceeds the number set as the spam barrier, SpamAssassin will not further process the message, and you will simply see the SpamTally: E-Mail header tag with the current score at the end.

D. spamassassin - SpamAssassin uses a variety of spam-detection techniques for INBOUND E-Mails, including DNS-based and fuzzy-checksum-based spam detection, Bayesian filtering, external programs, blacklists and online databases. Because this occurs at the INBOUND level, there WILL be a header entry if it is operational.

I learned that the tags I was used to see in the header from SpamAssassin that were missing is the new (to me) default behavior because the previously included header tags were messing up the Bayesian classification. The missing headers negated some of the rules I had set up in E-Mail clients to file incoming messages. ALL tags, and the only tags, added to the header by SpamAssassin begin with "X-Spam-". If it doesn't start with that, it is not from SpamAssassin. The following is the list of SpamAssassin header tags that are available, with the ones in blue being the ones that always show by default, and those that show in red additionally show up only when it is classified as spam, making it useful for categorizing on the client end if you pass spam to the client for final disposition:

1. X-Spam-Score: When trying to identify spam, SpamAssassin(tm) assigns a numeric score to each message. This score is listed in the X-Spam-Score: header for each message. For example: X-Spam-Score: 6.3

2. X-Spam-Checker-Version: The X-Spam-Checker-Version: header lists the version of SpamAssassin(tm) and the name of the server that was used to scan the message. This info is useful for system administrators, and can be ignored by end users.

3. X-Spam-Level: When trying to identify spam, SpamAssassin(tm) assigns a numeric score to each message. This numeric score is converted to a number of asterisks equal to the score of the message (rounded down), and added to the X-Spam-Level header. For example, a spam score of 3.6 would generate the following header: X-Spam-Level: ***

4. X-Spam-Status: The X-Spam-Status: is added to all messages and includes summary information including spam status (yes/no), score, what tests were triggered, and the SpamAssassin version. Here is an example:

X-Spam-Status: Yes, hits=6.2 required=5.0 tests=REVERSE_AGING,VIAGRA
autolearn=no version=2.64-servername_config_v2

In addition, email messages identified as spam also have these headers set:

5. X-Spam-Flag: This header is added if SpamAssassin(tm) identifies the message as being spam: 'X-Spam-Flag: YES'. You can use this message to filter spam in applications such as Outlook, Eudora, Thunderbird, etc. The tag only appears when the message is classified as spam, and thus will never appear as other than YES.

6. X-Spam-Report: When SpamAssassin(tm) scans a message it applies a series of "tests" to the message. Each test has a score. If the cumulative score for a message is over a specified number (5 is the default) then the message is identified as spam. When a message is identifed as being spam, SpamAssassin(tm) adds the X-Spam-Report header to the message. This header lists all of the tests that matched, and their corresponding scores. See http://www.spamassassin.org/tests.html for a list of tests (and their corresponding scores) that SpamAssassin(tm) uses. Here is an example X-Spam-Report header:
X-Spam-Report:
* 0.3 RCVD_NUMERIC_HELO Received: contains a numeric HELO
* 4.3 REVERSE_AGING BODY: Reverses Aging
* 1.9 VIAGRA BODY: Plugs Viagra

Analysis of my E-Mail header based on the my previous post:
Easy Spam Fighter - Inserted these tags into the E-Mail header.
Code:
[B]Forward-Confirmed-ReverseDNS:[/B] Reverse and forward lookup success on 57.128.088.184, -10 Spam score
[B]SPFCheck:[/B] Server passes SPF test, -30 Spam score
SpamAssassin - Inserted these tags into the E-Mail header.
Code:
[B]X-Spam-Score:[/B] 0.0 (/)
[B]X-Spam-Report:[/B] Spam detection software, running on the system "server.ReceivingEnd.com",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  <http://www.SendingEnd.com/> My Business,
    LLC Hi, Test Message text goes here Sincerely yours, cid:image002.gif@01D47B26.19F50A10
    John.Smith My Business, LLC Phone (789)123-456 <http://www.SendingEnd.com>
    http://www.SendingEnd.com [...] 
 Content analysis details:   (0.0 points, 5.0 required)
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
  0.0 URIBL_BLOCKED          ADMINIS
TRATOR NOTICE: The query to URIBL was
                             blocked.  See
                             http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
                              for more information.
                             [URIs: SendingEnd.com]
 -0.0 SPF_PASS               SPF: sender matches SPF record
  0.0 HTML_IMAGE_RATIO_06    BODY: HTML has a low ratio of text to image
                             area
  0.0 HTML_MESSAGE           BODY: HTML included in message
Since X-Spam-Report: will ALWAYS be in the header if SpamAssassin has scanned it, if it is missing on any E-Mail, it is a signal that E-Mails are not being scanned and you need to restart spamd, which probably would not interfere with downloading E-Mails, or exim which would, but would restart the SpamBlocker chain. Also notice that SpamAssassin doesn't have access to at least one URIBL as is indicated by the "ADMINISTRATOR NOTICE:" quote, which could have a large impact on its effectiveness. That is commonly an issue stemming from using a public DNS server that it has received a lot of traffic from and shut it off (as was the case), or the resolver is on a private IP address. This did not happen on the old server with and older SpamAssassin, which leads me to believe the current SpamAssassin configuration uses different URIBLs, that at least one doesn't like coming from the public DNS servers I had as primary and secondary. Making he server's own local BIND DNS primary fixed the problem instantly. The "ADMINISTRATOR NOTICE:" will prove handy to scan incoming E-Mails for to be alert me to problems.

Easy Spam Fighter - Inserted this tag into the E-Mail header.
Code:
[B]SpamTally:[/B] Final spam score: -40
Anti-virus results - SpamBlocker inserts the results of ClamAV scan into E-Mail header.
Code:
[B]X-Antivirus-Scanner:[/B] Clean mail though you should still use an Antivirus
If this tag is not present in the header, E-Mails are not being scanned for viruses, thus, you should scan for evidence of it.

POPFile - POPFile E-Mail Classification added to E-Mail header.
Code:
[B]X-Text-Classification:[/B] other
The buckets defined are other (ham), spam, and unclassified. The E-Mail client, moves the E-Mails to the spam folder based on based on the X-Text-Classification from POPFile's excellent Bayesian calculations, and/or SpamAssassin's has X-SPAM-Flag: being found in the header.

Other: What is conspicuously absent is a method to determine if the the OUTBOUND SMTP-time ACL blocking improvements and BlockCracking are functional without creating a real situation that puts you on a blacklist. I don't see anywhere where you can see live statistics to have any confidence that thing are working.
 
Last edited:
Top