Exim 4.93 has been released

ditto

Verified User
Joined
Apr 27, 2009
Messages
2,513
@Wanabo, it seems DirectAdmin has changed custombuild back to version 4.92.3, so that is the reason you have the previous version now.
 

annc

Verified User
Joined
Jun 10, 2008
Messages
36
Same issue here - after upgrade exim to 4.93 (3 our machines) many SPF failes even if SPF is correct. Terrible.
I've just downgraded to 4.92, I hope that will help
 

wattie

Verified User
Joined
May 31, 2008
Messages
1,041
Location
Bulgaria
s-nail: Fixed. It wanted gmake instead of make. CB2 rev 2315 released with the fix.
John
There is one more issue on FreeBSD. Now when I built Exim 4.92.3 I got the following error:

Code:
Exim installation complete
Moving exim binary.
./build: cannot create /etc/logrotate.d/exim: No such file or directory
./build: cannot create /etc/logrotate.d/exim: No such file or directory
./build: cannot create /etc/logrotate.d/exim: No such file or directory
./build: cannot create /etc/logrotate.d/exim: No such file or directory
On FreeBSD we do not use logrotate but newsyslog
 

DirectAdmin Support

Administrator
Staff member
Joined
Feb 27, 2003
Messages
8,961
I've added 4.93.0.2 to files1, but not yet updated the versions.txt.
For those affected, please run:
Code:
./build update
perl -pi -e 's/^exim:.*$/exim:4.93.0.2:/' versions.txt
./build exim
then confirm if it's updated, and if it has SPF:
Code:
exim -bV
and confirm back here. If it's fixed in this version, I'll update the versions.txt with it.

I've tested it here, but it works fine on our production test box, so not yet sure where the issue lies (ticket could be created for it if it's still not working)

Regarding FreeBSD+/etc/lograte.d/exim, Martynas has added a fix to CB so it doesn't show that anymore.
Will add the update to newsyslog.conf when able.

John
 
Last edited:

nmb

Verified User
Joined
Sep 13, 2008
Messages
208
Just tested it

# exim -bV
Exim version 4.93.0.2 #5 built 13-Dec-2019 08:51:33
Copyright (c) University of Cambridge, 1995 - 2018
(c) The Exim Maintainers and contributors in ACKNOWLEDGMENTS file, 2007 - 2018
Berkeley DB: Berkeley DB 5.3.21: (May 11, 2012)
Support for: crypteq IPv6 Perl OpenSSL move_frozen_messages Content_Scanning DKIM DNSSEC Event OCSP PRDR SPF TCP_Fast_Open Experimental_SRS
Lookups (built-in): lsearch wildlsearch nwildlsearch iplsearch cdb dbm dbmjz dbmnz dnsdb
Authenticators: cram_md5 dovecot plaintext spa
Routers: accept dnslookup ipliteral manualroute queryprogram redirect
Transports: appendfile/maildir/mailstore/mbx autoreply lmtp pipe smtp
Malware: f-protd f-prot6d drweb aveserver fsecure kavdaemon sophie clamd mksd avast sock cmdline
Fixed never_users: 0
Configure owner: 0:0
Size of off_t: 8
2019-12-13 08:51:56 cwd=/usr/local/directadmin/custombuild 2 args: exim -bV
Configuration file is /etc/exim.conf
Still didn't work :

2019-12-13 08:51:50 SPFCheck: 34.209.113.130 is not allowed to send mail from verifier.port25.com: Please see http://www.open-spf.org/Why?id=auth-results@verifier.port25.com&ip=34.209.113.130&receiver=xxx.xxx.xxx : Reason: mechanism
2019-12-13 08:51:50 H=verifier.port25.com [34.209.113.130] X=TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256 CV=no F=<auth-results@verifier.port25.com> rejected RCPT <xxx@xxx.com>: SPF: 34.209.113.130 is not allowed to send mail from verifier.port25.com: Please see http://www.open-spf.org/Why?id=auth-results@verifier.port25.com&ip=34.209.113.130&receiver=xxx.xxx.com : Reason: mechanism
 

annc

Verified User
Joined
Jun 10, 2008
Messages
36
Confirmed, 4.93.0.2 - SPF still doesn't work as it should (tested on Centos 7 with latest updates from yum)
 

DirectAdmin Support

Administrator
Staff member
Joined
Feb 27, 2003
Messages
8,961
Thanks for the feedback. Exim 4.93.0.3 has now been released, and I've again not updated the versions.txt until we can solve the issue.
Code:
cd /usr/local/directadmin/custombuild
./build update
perl -pi -e 's/^exim:.*$/exim:4.93.0.3:/' versions.txt
./build exim
For anyone affected by the issue, if you're able to provide root login in a ticket, that would also speed up the process.

The other possibility is that something is missing, preventing the SPF option from being added to exim's Local/Makefile before the build.
When you run ./build exim, if you can ctrl-z during the compile phase, before it finishes, and check the exim-4.93.0.3/Local/Makefile at the very bottom, to confirm it has
Code:
SUPPORT_SPF=yes
CFLAGS  += -I/usr/local/include
LDFLAGS += -lspf2
as if that's missing, it might mean that /usr/local/include/spf2/spf.h does not exist, which is the check CB uses to add the code to the Makefile so it's included.
Once done type "fg" to continue the build, and ctrl-c if you wish to abort.

John
 

DirectAdmin Support

Administrator
Staff member
Joined
Feb 27, 2003
Messages
8,961
Important to add, that SPF is part of the easy_spam_fighter module.. so that must be enabled first, prior to the exim compile if you're looking to have SPF compiled in:
Code:
./build set easy_spam_fighter yes
./build exim
./build exim_conf
Note... it also requires either SpamAssassin or Rspamd to be installed, as ESF uses them for realtime smtp spam blocking.

If you've confirmed that /usr/local/include/spf2/spf.h does not exist, you can alternatively test just that compile with:
Code:
./build libspf2
to debug the addition of the spf.h, but must follow it up with the ./build exim after, to get it compiled in.
 

nmb

Verified User
Joined
Sep 13, 2008
Messages
208
Tested Exim 4.93.0.3 and still has same problem. (Rspamd is installed and easy_spam_fighter is set to yes)

Below is exim-4.93.0.3/Local/Makefile :

Code:
# ENABLE_DISABLE_FSYNC=yes
USE_OPENSSL=yes
SUPPORT_SPF=yes
CFLAGS  += -I/usr/local/include
LDFLAGS += -lspf2
CFLAGS  += -fstack-protector --param ssp-buffer-size=4
EXPERIMENTAL_SRS=yes
LDFLAGS += -lsrs_alt
 

annc

Verified User
Joined
Jun 10, 2008
Messages
36
The problem is not that SPF doesn't work. It works (so nothing is missing...). But it gives us many many false-positive hits.
And again - not all of the mails are false-positive - only some of them, but without any sensible reason.
 

DirectAdmin Support

Administrator
Staff member
Joined
Feb 27, 2003
Messages
8,961
Tested Exim 4.93.0.3 and still has same problem. (Rspamd is installed and easy_spam_fighter is set to yes)

Below is exim-4.93.0.3/Local/Makefile :

Code:
# ENABLE_DISABLE_FSYNC=yes
USE_OPENSSL=yes
SUPPORT_SPF=yes
CFLAGS  += -I/usr/local/include
LDFLAGS += -lspf2
CFLAGS  += -fstack-protector --param ssp-buffer-size=4
EXPERIMENTAL_SRS=yes
LDFLAGS += -lsrs_alt
Ok, if that's enabled, but exim -bV is not showing "SPF" in the "Support for:" line, then I'm not sure... and would likely need to login to debug this. (eg: ticket)

----
@annc: Can you paste us the exact SPF error message from the mainlog? I'm looking to verify the true domain, so perhaps a ticket/email might be best. Or.. if you can run "dig TXT domain.com" to see what the current spf records are for the blocked domain, then you can cross reference that with the actual incoming IP address to see for yourself what it is, if it should be blocked, and if exim is way off or not. I am suspecting something with their IPv6 support change for SPF

====
A workaround for the full spf=fail drop, the score can be lowered, so it's only added to the total, instead of fully dropped:
Just the EASY_SPF_FAIL==60 would be needed.. which will not drop, as long as it's below EASY_HIGH_SCORE_DROP, which defaults to 100.
So setting EASY_SPF_FAIL==60 means it still get's flagged, but only drops if that 60 plus everything else it found (eg: rdns, etc) is over 100.

But we'll try and find the actual source of the SPF issue, since that's the problem at the moment.

John
 

annc

Verified User
Joined
Jun 10, 2008
Messages
36
OK, I've just sent you this log and additional information to support@directadmin.com (subject of my mail: "SPF Failed - answer for forum ticket").
I don't want to paste in public forum that informations :)

We've got about one hundred domain false-positive SPF (even.... microsoft.com :)) in logs after upgrade to 4.93 so.... I'm your guy 🙃
 
Last edited:

ditto

Verified User
Joined
Apr 27, 2009
Messages
2,513
Important to add, that SPF is part of the easy_spam_fighter module.. so that must be enabled first, prior to the exim compile if you're looking to have SPF compiled in: [cut]
I think I must misunderstand you, if not this statement really confuses me. As we do not yet use ESF, and do not have it enabled on any servers. We only run plain SpamAssassin. But you seem to say that we need ESF enabled for SPF to work. However in all those years we have not had any trouble with SPF, and it does work without ESF (we are currently running Exim 4.92.3). Can you please clearify if it now suddenly is required to have ESF enabled in order for SPF in to work correctly when sending emails?
 

DirectAdmin Support

Administrator
Staff member
Joined
Feb 27, 2003
Messages
8,961
I've done more digging based on a few things and found the cause, but don't have a solution as the change by exim is fairly major, so they'd need to offer a fix (assuming my findings are accurate)

If an spf TXT record has "ip4:1.1.2.3.4" the check works correctly, and the value can "pass", since no A/MX lookups are done if the IP is explicitly set.

However, if the TXT record does not have an explicit IP, and relies on something like
Code:
v=spf a mx -all
for example, the new lookup code they've added seems to completely fail. As previously guessed, the suspect "support for IPv6 SPF lookups" change theory is supported by the changes I'm seeing in exim-4.93/src/lookups/spf.c, from the 4.92.3 version of the same file (might be elsewhere, but that's a ballpark guess). There are several reference to HAVE_IPV6, which makes sense.. and we do include HAVE_IPV6=yes by default.... so the new changes that were added are included in this build.

I did add "-DSPF" to the CFLAGS, just in case it applies here, as I found some other guide making reference to it ... but I couldn't find any reference to, but worth a shot anyway.

So as far as I can tell, this is new exim bug, introduced in 4.93, with the addition of the IPv6 SPF check support.. but the result is that the "a mx" type lookups simply don't do anything. Any TXT spf records that include the ip4:1.2.3.4 style checks should work fine.

I realized there are plenty of debug points in that code, so ran exim in debug mode:
Code:
exim -d -bd -oP /var/run/exim.pid
and the SPF debug code showed itself failing the A/MX lookups, when they definitely should not have been bee failing (I've swapped the domain in question and sending IP out here, for privacy reasons, but the A and MX records definitely resolve just fine). Key items have #commants on the right:
Code:
...
7234 check spf = fail                                                            #this is the condition check which triggers the failure, below
7234 spf_process
spf_dns.c:54         Debug: DNS[cache] lookup: testdomain.ca SPF (99)
spf_dns.c:54         Debug: DNS[exim] lookup: testdomain.ca SPF (99)
7234 SPF_dns_exim_lookup
7234 DNS lookup of testdomain.ca (SPF) gave NO_DATA
7234 returning DNS_NODATA
7234 faking res_search(SPF) response length as 65535
7234  writing neg-cache entry for testdomain.ca-SPF-880041, ttl 86400
spf_dns.c:66         Debug: DNS[exim] found record
spf_dns.c:69         Debug:     DOMAIN: (null)  TYPE: ANY (255)
spf_dns.c:76         Debug:     TTL: 86400  RR found: 0  herrno: 1  source: exim
spf_dns.c:66         Debug: DNS[cache] found record
spf_dns.c:69         Debug:     DOMAIN: (null)  TYPE: ANY (255)
spf_dns.c:76         Debug:     TTL: 86400  RR found: 0  herrno: 1  source: exim
spf_server.c:356     Debug: get_record(testdomain.ca): HOST_NOT_FOUND
spf_dns.c:54         Debug: DNS[cache] lookup: testdomain.ca TXT (16)
spf_dns.c:54         Debug: DNS[exim] lookup: testdomain.ca TXT (16)
7234 SPF_dns_exim_lookup
7234 DNS lookup of testdomain.ca (TXT) succeeded
spf_dns.c:66         Debug: DNS[exim] found record
spf_dns.c:69         Debug:     DOMAIN: testdomain.ca  TYPE: TXT (16)
spf_dns.c:76         Debug:     TTL: 100  RR found: 1  herrno: 0  source: exim
spf_dns.c:94         Debug:     - TXT: v=spf1 a mx -all           #this is correct, as set for testing
spf_dns.c:66         Debug: DNS[cache] found record
spf_dns.c:69         Debug:     DOMAIN: testdomain.ca  TYPE: TXT (16)
spf_dns.c:76         Debug:     TTL: 100  RR found: 1  herrno: 0  source: exim
spf_dns.c:94         Debug:     - TXT: v=spf1 a mx -all
spf_server.c:400     Debug: get_record(testdomain.ca): NETDB_SUCCESS
spf_server.c:441     Debug: found SPF record: v=spf1 a mx -all
spf_compile.c:1210   Debug: Compiling record v=spf1 a mx -all
spf_compile.c:1314   Debug: Name starts at  a mx -all
spf_compile.c:1408   Debug: Adding mechanism type 1
spf_compile.c:847    Debug: SPF_c_mech_add: type=1, value= mx -all
spf_compile.c:1314   Debug: Name starts at  mx -all
spf_compile.c:1408   Debug: Adding mechanism type 2
spf_compile.c:847    Debug: SPF_c_mech_add: type=2, value= -all
spf_compile.c:1314   Debug: Name starts at  all
spf_compile.c:1408   Debug: Adding mechanism type 8
spf_compile.c:847    Debug: SPF_c_mech_add: type=8, value=
spf_dns.c:54         Debug: DNS[cache] lookup: testdomain.ca A (1)
spf_dns.c:54         Debug: DNS[exim] lookup: testdomain.ca A (1)
7234 SPF_dns_exim_lookup
7234 DNS lookup of testdomain.ca (A) succeeded                                #this is a good thing, I assume.
spf_dns.c:66         Debug: DNS[exim] found record
spf_dns.c:69         Debug:     DOMAIN: (null)  TYPE: ANY (255)             #DOMAIN==NULL?  Might be correct.. might be wrong.
spf_dns.c:76         Debug:     TTL: 86400  RR found: 0  herrno: 1  source: exim
spf_dns.c:66         Debug: DNS[cache] found record
spf_dns.c:69         Debug:     DOMAIN: (null)  TYPE: ANY (255)
spf_dns.c:76         Debug:     TTL: 86400  RR found: 0  herrno: 1  source: exim
spf_interpret.c:783  Debug: found 0 A records for testdomain.ca  (herrno: 1)    #FALSE: there are definitely A records
spf_dns.c:54         Debug: DNS[cache] lookup: testdomain.ca MX (15)
spf_dns.c:54         Debug: DNS[exim] lookup: testdomain.ca MX (15)
7234 SPF_dns_exim_lookup
7234 DNS lookup of testdomain.ca (MX) succeeded
spf_dns.c:66         Debug: DNS[exim] found record
spf_dns.c:69         Debug:     DOMAIN: (null)  TYPE: ANY (255)
spf_dns.c:76         Debug:     TTL: 86400  RR found: 0  herrno: 1  source: exim
spf_dns.c:66         Debug: DNS[cache] found record
spf_dns.c:69         Debug:     DOMAIN: (null)  TYPE: ANY (255)
spf_dns.c:76         Debug:     TTL: 86400  RR found: 0  herrno: 1  source: exim
spf_interpret.c:824  Debug: found 0 MX records for testdomain.ca  (herrno: 1)
7234 SPF result is fail (3)
7234   message: SPF: $sender_host_address is not allowed to send mail from $sender_address_domain: $spf_smtp_comment
7234 check logwrite = SPFCheck: $sender_host_address is not allowed to send mail from $sender_address_domain: $spf_smtp_comment
7234                = SPFCheck: 1.2.3.4 is not allowed to send mail from testdomain.ca: Please see http://www.open-spf.org/Why?id=john%40testdomain.ca&ip=1.2.3.4&receiver=server.testdomain.ca : Reason: mechanism
Anyway, this isn't a solution, just tracking down the cause of the SPF failure to be somewhere in there.
Hopefully this debug output will help the exim devs track it down.

For the time being, please stick with 4.92.3, as is set in the versions.txt.

@ditto: To clarify, in you need SpamAssassin in order for ESF to work.
The ESF has it's own SPF checks if you want to fully block based on an spf=fail, regardless of a summed score that SpamAssassin provides.
SPF checks can still be done by SpamAssassin, without ESF turned on, but gets scored instead of a hard block.
 

DirectAdmin Support

Administrator
Staff member
Joined
Feb 27, 2003
Messages
8,961
For comparison, the same debug mode with 4.92.3 gives us this, far shorter successful output
Code:
14238 DNS lookup of testdomain.ca (TXT) succeeded
14238 lookup yielded: v=spf1 a mx -all
14238 check set acl_m_spf_record = ${lookup dnsdb{txt=$sender_address_domain}{$value}}
14238                            = v=spf1 a mx -all
14238 check logwrite = acl_m_spf_record TXT for $sender_address_domain: $acl_m_spf_record
14238                = acl_m_spf_record TXT for testdomain.ca: v=spf1 a mx -all
14238 LOG: MAIN
14238   acl_m_spf_record TXT for testdomain.ca: v=spf1 a mx -all
14238 warn: condition test succeeded in ACL "acl_check_recipient"
14238 processing "drop"
14238 check !authenticated = *
14238 check condition = ${if !eq{$acl_c_esf_skip}{1}}
14238                 = true
14238 check condition = ${if >={100}{100}}
14238                 = true
14238 check spf = fail
14238 SPF result is pass (2)
 

ikkeben

Verified User
Joined
May 22, 2014
Messages
691
Location
Netherlands Germany
I've done more digging based on a few things and found the cause, but don't have a solution as the change by exim is fairly major, so they'd need to offer a fix (assuming my findings are accurate)

If an spf TXT record has "ip4:1.1.2.3.4" the check works correctly, and the value can "pass", since no A/MX lookups are done if the IP is explicitly set.

However, if the TXT record does not have an explicit IP, and relies on something like
Code:
v=spf a mx -all
for example, the new lookup code they've added seems to completely fail. As previously guessed, the suspect "support for IPv6 SPF lookups" change theory is supported by the changes I'm seeing in exim-4.93/src/lookups/spf.c, from the 4.92.3 version of the same file (might be elsewhere, but that's a ballpark guess). There are several reference to HAVE_IPV6, which makes sense.. and we do include HAVE_IPV6=yes by default.... so the new changes that were added are included in this build.

I did add "-DSPF" to the CFLAGS, just in case it applies here, as I found some other guide making reference to it ... but I couldn't find any reference to, but worth a shot anyway.

So as far as I can tell, this is new exim bug, introduced in 4.93, with the addition of the IPv6 SPF check support.. but the result is that the "a mx" type lookups simply don't do anything. Any TXT spf records that include the ip4:1.2.3.4 style checks should work fine.
Maybe EXIM followed this advise ? ( partly or some developers..:rolleyes: )

Use of the “a” and “mx” directives§ The use of a and mx directives in SPF records is no longer recommended.
Don't know also if they support MX null record after some update. .. offtopic maybe then sorry

Also don't know if this is still used in 4.93.x https://github.com/Exim/exim/wiki/SPF
SPF is specified in https://tools.ietf.org/html/rfc7208.

SPF support is added via the libspf2 library. Visit


Here readers can find some of exim see dnslookup router and some there. https://www.exim.org/exim-html-current/doc/html/spec_html/index.html

For the lookup this https://www.exim.org/exim-html-current/doc/html/spec_html/ch-the_dnslookup_router.html#SECTprowitdnsloo

another edit if it is only a problem when ipv6 then consider how exim handles MX if a kind of error with those then maybe the sorted order is important:
When a host has more than one IP address, they are sorted into a random order, except that IPv6 addresses are sorted before IPv4 addresses. If all the IP addresses found are discarded by a setting of the ignore_target_hosts generic option, the router declines.
For example sofar i know none Microsoft SPF included has IPV6 in the spf record itself and also none MX or A , so i don't understand if this one is still problem?
 
Last edited:

annc

Verified User
Joined
Jun 10, 2008
Messages
36
Good and nice debug job John :)

Hope exim's Devs are going to help and fix this bug soon
 
Top