View Poll Results: Would you like DA to include DMARC Analysis and Reporting?

Voters
20. You may not vote on this poll
  • Yes

    20 100.00%
  • No

    0 0%
  • Don't care

    0 0%
Results 1 to 19 of 19

Thread: Integrate OpenDMARC with Exim to have DMARC checked and report sent out

  1. #1
    Join Date
    Oct 2004
    Location
    London, UK
    Posts
    6,739

    Integrate OpenDMARC with Exim to have DMARC checked and report sent out

    Hi everyone,
    in last few days I have been playing around with DMARC because I wanted to be able to check it, and report it as many providers are already doing, so I started to work on it using OpenDMARC.

    Here it is a small How-To that will help you to make this integration, depending on the feedback of this How-To DirectAdmin Staff will decide if include this in CB to allow manage it that way.

    This guide is based on CentOS 7, it may be very similar on other OS but I have not them to play with, hopefully someone will want to replicate this guide in Debian and post it here.

    NOTE: I am setting in the command a DB password as "PASSWORD", you MUST change it!

    First bit, you need to remove sendmail* from the /etc/yum.conf file on line exclude=
    Once done let's proceed to install opendmarc and all it's dependencies
    Code:
    yum -y install opendmarc libopendmarc-devel
    Configuration file: /etc/opendmarc.conf
    I am using this one: https://repository.crazynetwork.it/d...opendmarc.conf
    But you can easly edit it yourself, the important option to set is HistoryFile which should be set to /var/spool/exim/opendmarc.dat


    Now we need to customize the exim Makefile to enable two Experimental features (SPF and DMARC) so we start downloading the standard one that DA use and store it in the CB custom folder:
    Code:
    mkdir -p /usr/local/directadmin/custombuild/custom/exim/
    wget -O /usr/local/directadmin/custombuild/custom/exim/Makefile http://files.directadmin.com/services/custombuild/Makefile
    And we will edit it this way:
    Look for EXPERIMENTAL_SPF=yes
    And uncomment it and the line just after, like this:
    HTML Code:
    EXPERIMENTAL_SPF=yes
    CFLAGS  += -I/usr/local/include
    Than, at the end of all the experimental options add:
    Code:
    # Uncomment the following lines to add DMARC support, you must first enable SPF.
    # Please read the Local/Makefile comments on enabling the SUPPORT_SPF
    # feature.  You must also have DKIM support, so you cannot set the
    # DISABLE_DKIM feature.  Once both of those conditions have been met
    # you can enable DMARC in Local/Makefile:
    
    EXPERIMENTAL_DMARC=yes
    LDFLAGS += -lopendmarc
    (Or you can download the ready version here: https://repository.crazynetwork.it/d.../exim/Makefile using wget -O /usr/local/directadmin/custombuild/custom/exim/Makefile https://repository.crazynetwork.it/d.../exim/Makefile using

    Once done, let's build exim
    Code:
    /usr/local/directadmin/custombuild/build exim
    Now, let's do some configurations files:
    Create this file (or update it to also contain the following): /etc/exim.acl_check_recipient.pre.conf
    Code:
    warn    authenticated   = *
            hosts           = +relay_hosts
            domains         = +local_domains
            control         = dmarc_disable_verify
    
    warn    !authenticated  = *
            hosts           = !+relay_hosts
            domains         = !+local_domains
            control         = dmarc_enable_forensic
    Now we need the TLDs DB from https://publicsuffix.org/
    Code:
    wget -O /etc/exim_opendmarc.tlds https://publicsuffix.org/list/public_suffix_list.dat
    And configure the spool and the TLDs file in exim too in:
    Code:
    touch /etc/exim.variables.conf.custom
    echo "dmarc_history_file=/var/spool/exim/opendmarc.dat" >> /etc/exim.variables.conf.custom
    echo "dmarc_tld_file=/etc/exim_opendmarc.tlds" >> /etc/exim.variables.conf.custom
    And rebuild exim configurations.
    Code:
    /usr/local/directadmin/custombuild/build exim_conf
    Add the following at the end of the file: /etc/exim.easy_spam_fighter/check_message.conf.custom.post
    Code:
    # DMARC Checks
      warn
        dmarc_status         = accept : none : off
        !authenticated       = *
        log_message          = DMARC DEBUG: $dmarc_status $dmarc_used_domain
    
      warn
        dmarc_status         = !accept
        !authenticated       = *
        log_message          = DMARC DEBUG: '$dmarc_status' for $dmarc_used_domain
    
      warn
        dmarc_status         = quarantine
        !authenticated       = *
        set acl_m_quarantine = 1
        # Do something in a transport with this flag variable
    
      deny
        condition            = ${if eq{$dmarc_domain_policy}{reject}}
        condition            = ${if eq{$acl_m_mailing_list}{1}}
        message              = Messages from $dmarc_used_domain break mailing lists
    
      deny
        dmarc_status         = reject
        !authenticated       = *
        message              = Message from $dmarc_used_domain failed sender's DMARC policy, REJECT
    
      warn
        add_header           = :at_start:${authresults {$primary_hostname}}
    Now restart exim
    Code:
    service exim restart
    And you should be able to see in a DMARC in the exim logs (try to make some traffic from a site that have DMARC):
    tail -f /var/log/exim/mainlog | grep DMARC

    And you should also be able to see a DMARC line in the headers:
    Code:
    Authentication-Results: Orange01.CrazyNetwork.it; dmarc=pass header.from=gmail.com
    Now, let's send out some reports!
    To do this you will need:
    A MySQL DB
    An email address that will send out email (with SMTP Auth)
    2 Scripts (one is opendmarc-report which I have modified to use SMTP Auth when sending out emails)

    Prepare the MySQL DB:
    Code:
    wget -O /tmp/da_opendmarc.sql https://repository.crazynetwork.it/opendmarc/da_opendmarc.sql
    /usr/bin/mysql -uda_admin -p`grep "^passwd=" $DA_PATH/conf/mysql.conf | cut -d= -f2` -e "CREATE DATABASE da_opendmarc"
    /usr/bin/mysql -uda_admin -p`grep "^passwd=" $DA_PATH/conf/mysql.conf | cut -d= -f2` -e "CREATE USER 'da_opendmarc'@'127.0.0.1' IDENTIFIED BY 'PASSWORD';"
    /usr/bin/mysql -uda_admin -p`grep "^passwd=" $DA_PATH/conf/mysql.conf | cut -d= -f2` -e "GRANT ALL PRIVILEGES ON da_opendmarc . * TO 'da_opendmarc'@'127.0.0.1';"
    /usr/bin/mysql -uda_admin -p`grep "^passwd=" $DA_PATH/conf/mysql.conf | cut -d= -f2` -e "FLUSH PRIVILEGES;"
    mysql -uda_opendmarc -pPASSWORD da_opendmarc < /tmp/da_opendmarc.sql
    rm -rf /tmp/da_opendmarc.sql
    Get the scripts:
    Code:
    wget -O /usr/local/directadmin/scripts/custom/exim_dmarc.sh https://repository.crazynetwork.it/directadmin/scripts/custom/exim_dmarc.sh
    wget -O /usr/local/directadmin/scripts/custom/opendmarc-reports https://repository.crazynetwork.it/directadmin/scripts/custom/opendmarc-reports
    chmod 700 /usr/local/directadmin/scripts/custom/exim_dmarc.sh 
    chmod 700 /usr/local/directadmin/scripts/custom/opendmarc-reports
    Edit /usr/local/directadmin/scripts/custom/exim_dmarc.sh and fill the required DB and SMTP Auth informations changing those lines:
    Code:
    DBHOST="localhost"
    DBNAME="da_opendmarc"
    DBUSER="da_opendmarc"
    DBPASS="DATABASE PASSWORD"
    REPORTEMAIL="EMAIL USERNAME"
    REPORTSMTP_HOST="EMAIL SERVER HOST"
    REPORTSMTP_PORT="EMAIL SERVER PORT"
    REPORTSMTP_USER="EMAIL AUTH USER"
    REPORTSMTP_PASS="EMAIL AUTH PASS"
    REPORTORG="REPORTER NAME"
    And finally, add the required cronjobs:
    Code:
    echo "0 * * * * /usr/local/directadmin/scripts/custom/exim_dmarc.sh -i >/dev/null 2>&1" >> /etc/cron.d/directadmin_cron
    echo "30 */6 * * * /usr/local/directadmin/scripts/custom/exim_dmarc.sh -r >/dev/null 2>&1" >> /etc/cron.d/directadmin_cron
    echo "0 0 0 * * /usr/local/directadmin/scripts/custom/exim_dmarc.sh -e >/dev/null 2>&1" >> /etc/cron.d/directadmin_cron

    Now everything should be configured.
    A suggestion would be, before adding the cronjob to test everything (for example sending from one of you domain to another of your domain a mail which is DMARC signed so to check everything)

    IMPORT DMARC DATA INTO DB: /usr/local/directadmin/scripts/custom/exim_dmarc.sh -i
    SEND DMARC REPORTS OUT: /usr/local/directadmin/scripts/custom/exim_dmarc.sh -r
    DELETE OLD DATA FROM DB: /usr/local/directadmin/scripts/custom/exim_dmarc.sh -E
    SHOW THE REPORT THAT ARE GOING TO BE SENT (NO SEND): /usr/local/directadmin/scripts/custom/exim_dmarc.sh -t

    I hope that this will be helpful to someone, if I forgot something or you face any problem let me know and I will try to help

    Best regards
    Last edited by SeLLeRoNe; 04-30-2018 at 02:35 AM.
    SeLLeRoNe - Andrea Iannucci
    Head of Managed Service - Senior DevOps Engineer
    If you need my support write me an E-Mail to Support@CrazyNetwork.it

  2. #2
    Join Date
    Dec 2005
    Location
    The Netherlands
    Posts
    86
    Nice, very nice! DA should include this.

  3. #3
    Join Date
    Apr 2005
    Location
    Melbourne / Australia
    Posts
    29
    Awesome work Andrea, Directadmin this is a must do.

  4. #4
    Join Date
    Apr 2005
    Location
    GMT +7.00
    Posts
    12,133
    Hello Andrea,

    Thanks for sharing. I guess it's a typo:

    Code:
    "CREATE DATABASE da_opendmarc.sql"


    and you don't need .sql here.


    And probably add a note to change
    PASSWORD to a real password, otherwise they will use it as a password

  5. #5
    Join Date
    Oct 2004
    Location
    London, UK
    Posts
    6,739
    Hi Alex,
    yep, typo

    Fixed!

    Thanks for noticing
    SeLLeRoNe - Andrea Iannucci
    Head of Managed Service - Senior DevOps Engineer
    If you need my support write me an E-Mail to Support@CrazyNetwork.it

  6. #6
    Join Date
    Oct 2004
    Location
    London, UK
    Posts
    6,739
    DirectAdmin have now introduced /etc/exim.easy_spam_fighter/check_message.conf.custom.post and /etc/exim.easy_spam_fighter/check_message.conf.custom.pre

    Updating the original post to use the right file which will not be overwritten by CB
    SeLLeRoNe - Andrea Iannucci
    Head of Managed Service - Senior DevOps Engineer
    If you need my support write me an E-Mail to Support@CrazyNetwork.it

  7. #7
    Join Date
    Dec 2005
    Location
    The Netherlands
    Posts
    86
    Nice, that helps a lot. Is DA interested in adding this feature into DirectAdmin (exim)?

  8. #8
    Join Date
    Oct 2004
    Location
    London, UK
    Posts
    6,739
    They would prefer to not introduce experimental feature in production enviornments which is fair enough.
    But if the request for this is high enough they will add this as optional thing in CB I guess.

    Best regards
    SeLLeRoNe - Andrea Iannucci
    Head of Managed Service - Senior DevOps Engineer
    If you need my support write me an E-Mail to Support@CrazyNetwork.it

  9. #9
    Join Date
    May 2014
    Posts
    78
    I believe that DirectAdmin should simply add this option. Then you can make your own choice.
    Other control panels such as Plesk have already had this option.

    So I say, add DirectAdmin!

  10. #10
    Join Date
    Oct 2004
    Location
    London, UK
    Posts
    6,739
    With last release of exim (4.90.1) I am having troubles on compiling exim with DMARC.

    I will check into this tomorrow and I will update the thread once I mange to have it working again.
    SeLLeRoNe - Andrea Iannucci
    Head of Managed Service - Senior DevOps Engineer
    If you need my support write me an E-Mail to Support@CrazyNetwork.it

  11. #11
    Join Date
    Apr 2005
    Location
    GMT +7.00
    Posts
    12,133
    It builds fine on CentOS 6:

    Code:
    # exim -bV
    Exim version 4.90_1 #4 built 13-Feb-2018 12:10:56
    Copyright (c) University of Cambridge, 1995 - 2017
    (c) The Exim Maintainers and contributors in ACKNOWLEDGMENTS file, 2007 - 2017
    Berkeley DB: Berkeley DB 4.7.25: (March 22, 2017)
    Support for: crypteq IPv6 Perl OpenSSL move_frozen_messages Content_Scanning DKIM DNSSEC Event OCSP PRDR Experimental_SPF Experimental_SRS Experimental_DMARC
    Lookups (built-in): lsearch wildlsearch nwildlsearch iplsearch cdb dbm dbmjz dbmnz dnsdb
    Authenticators: cram_md5 dovecot plaintext spa
    Routers: accept dnslookup ipliteral manualroute queryprogram redirect
    Transports: appendfile/maildir/mailstore/mbx autoreply lmtp pipe smtp
    Fixed never_users: 0
    Configure owner: 0:0
    Size of off_t: 8
    with your guide.

    You can see in the output Experimental_DMARC

  12. #12
    Join Date
    Oct 2004
    Location
    London, UK
    Posts
    6,739
    True, apparently one of my server it isn't (same versions and OS), seems it doesn't seems to find dmarc.h

    Code:
    In file included from exim.h:515:0,
                     from macro_predef.c:11:
    dmarc.h:14:30: fatal error: opendmarc/dmarc.h: No such file or directory
     # include "opendmarc/dmarc.h"
    SeLLeRoNe - Andrea Iannucci
    Head of Managed Service - Senior DevOps Engineer
    If you need my support write me an E-Mail to Support@CrazyNetwork.it

  13. #13
    Join Date
    Oct 2004
    Location
    London, UK
    Posts
    6,739
    Okay a reinstall of opendmarc and libopendmarc fixed it.

    Strange that it was broken without any change...
    SeLLeRoNe - Andrea Iannucci
    Head of Managed Service - Senior DevOps Engineer
    If you need my support write me an E-Mail to Support@CrazyNetwork.it

  14. #14
    Join Date
    Oct 2004
    Location
    London, UK
    Posts
    6,739
    Alex, not sure you have noticed that but DA added a custom pre/post check message and I have updated the original post to use the post one instead of edit the CB mantianed one

    Use: /etc/exim.easy_spam_fighter/check_message.conf.custom.post
    Instead of: /etc/exim.easy_spam_fighter/check_message.conf
    SeLLeRoNe - Andrea Iannucci
    Head of Managed Service - Senior DevOps Engineer
    If you need my support write me an E-Mail to Support@CrazyNetwork.it

  15. #15
    Join Date
    Oct 2004
    Location
    London, UK
    Posts
    6,739
    The (EXPERIMENTAL_DMARC) variable $dmarc_ar_header is withdrawn, being replaced by the ${authresults } expansion in latest exim, so I have to update the check_message.conf.custom.post file, will update once I have a working fix, for now means that this is not working anymore and it actually block the email flow because of the deprecated variable.

    If anyone already noticed (and fixed) this, please let me know.

    Thanks
    SeLLeRoNe - Andrea Iannucci
    Head of Managed Service - Senior DevOps Engineer
    If you need my support write me an E-Mail to Support@CrazyNetwork.it

  16. #16
    Join Date
    Oct 2004
    Location
    London, UK
    Posts
    6,739
    Ok, I have found the required changes.

    I will update the first post shortly.
    SeLLeRoNe - Andrea Iannucci
    Head of Managed Service - Senior DevOps Engineer
    If you need my support write me an E-Mail to Support@CrazyNetwork.it

  17. #17
    Join Date
    May 2014
    Posts
    78
    Will DirectAdmin add this function by default? Other control panels already have it. (think of Plesk and a Synology mailserver NAS)

  18. #18
    Join Date
    Oct 2004
    Location
    London, UK
    Posts
    6,739
    Are you sure other already have this experimental functionality to send out DMARC reports?
    This is not DMARC signature is to validate the signature of incoming emails and send-out the reports with OpenDMARC.

    DA will probably implement this as soon as exim staff declare this functionality as not experimental anymore.
    SeLLeRoNe - Andrea Iannucci
    Head of Managed Service - Senior DevOps Engineer
    If you need my support write me an E-Mail to Support@CrazyNetwork.it

  19. #19
    Join Date
    May 2014
    Posts
    78
    Quote Originally Posted by SeLLeRoNe View Post
    Are you sure other already have this experimental functionality to send out DMARC reports?
    Yes, I do We manage Plesk servers and they send a message back. DMARC is also returned by a Synology NAS. (also in the logfiles)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •