How to make Exim send emails via Sendgrid [SOLVED]

Aspegic

Aspegic

Verified User
Joined
Aug 4, 2005
Messages
282
Because my provider (Google) is blocking connections to external port 25, I cannot send email the normal way with Exim. So I had no choice but to choose a third-party mail provider. I chose Sendgrid.

I have spend almost a day and a half now trying to get this to work.
I do not understand where it is going wrong.
I started with this:


That didn't work. I tried a zillion different variations. I read all different suggetions I could find searching the net. Not very much information is available b.t.w.. Most is for other control panels.

I downloaded the 3 files from the help.directadmin.com link.

This is what I have in: exim.authenticators.post.conf:

auth_login: driver = plaintext public_name = LOGIN client_send = : apikey : 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

I tried several different alternatives here, with the domain name instead of "apikey", with and without spaces around "apikey" and the key itself. With and without "hide". It all made no difference.
(Don't worry, the key is fake of course.)


This is what I have in: exim.routers.pre.conf

smart_route: driver = manualroute domains = ! +local_domains ignore_target_hosts = 127.0.0.0/8 condition = "${perl{check_limits}}" #use remote_smtp only if you do not need smtp-auth. #transport = remote_smtp #use auth_relay if you do need to set the remote smtpauth transport = auth_relay route_list = * smtp.sendgrid.net::587 no_more

Tried some alternatives here I found in other webpages, for example to use "byname" and puting the route_list between quotes. It all did not matter.


This is what I have in: exim.transports.pre.conf

auth_relay: driver = smtp hosts_require_auth = $host_address hosts_require_tls = $host_address headers_add = "${if def:authenticated_id{X-Authenticated-Id: ${authenticated_id}}}" interface = <; ${if exists{/etc/virtual/domainips}{${lookup{$sender_address_domain}lsearch*{/etc/virtual/domainips}}}} helo_data = ${if exists{/etc/virtual/helo_data}{${lookup{$sending_ip_address}iplsearch{/etc/virtual/helo_data}{$value}{$primary_hostname}}}{$primary_hostname}} hosts_try_chunking = hosts_try_fastopen = .include_if_exists /etc/exim.dkim.conf

Tried several different things here as well. With and without a port number (587). With and without hosts = but that did not make a difference either.


I am at a loss. WHAT are the correct settings for these 3 files? Or are there any other files I need to edit maybe? I made sure that the owner/group/permissions are the same as of exim.conf, so that should be correct.

If you want to know what error message I receive? Several different ones.

authentication required but authentication attempt(s) failed 535 Authentication failed: The provided authorization grant is invalid, expired, or revoked 535 Authentication failed: Bad username / password

all depending on which combination of settings I was trying. At one time I did not get an error message, but still no email was delivered. But in most cases the test mails just end up in the mail queue and eventually get frozen.

The domain has been verified in Sendgrid.
The proper CNAME records have been added to the DNS zone of my domain.
A certificate has been created using Letsencrypt and is valid for the domain.

SOLVED: for the solution see the settings in post #3 below.
 
Last edited:
I finally got it working.

I started with these instructions: https://help.directadmin.com/item.php?id=153
(actually the instructions on that page are incomplete, but I figured it out anyway)

I downloaded the 3 files and edited them as necessary, like this:

exim.authenticators.post.conf:

Code: 
auth_login:
    driver = plaintext
    public_name = LOGIN
    hide client_send = : apikey : SG.7CAOp5KjTlGoVrt4B-UI9A.HjcMbYGuGbiBIVjbb87VBHJ8eqA0w3zQ1VKNrqQM

(the key is fake of course)

exim.routers.pre.conf:

Code: 
smart_route:
     driver = manualroute
     domains = ! +local_domains
     ignore_target_hosts = 127.0.0.0/8
     condition = "${perl{check_limits}}"

     #use remote_smtp only if you do not need smtp-auth.
     #transport = remote_smtp

     #use auth_relay if you do need to set the remote smtpauth
     transport = auth_relay

     route_list = * smtp.sendgrid.net::587
     no_more

exim.transports.pre.conf:

Code: 
auth_relay:
    driver = smtp
    port = 25
    hosts_require_auth = $host_address
    hosts_require_tls = $host_address
    headers_add = "${if def:authenticated_id{X-Authenticated-Id: ${authenticated_id}}}"
    interface = <; ${if exists{/etc/virtual/domainips}{${lookup{$sender_address_domain}lsearch*{/etc/virtual/domainips}}}}
    helo_data = ${if exists{/etc/virtual/helo_data}{${lookup{$sending_ip_address}iplsearch{/etc/virtual/helo_data}{$value}{$primary_hostname}}}{$primary_hostname}}
    hosts_try_chunking =
    hosts_try_fastopen =
.include_if_exists /etc/exim.dkim.conf

I restarted Exim and now it works.

Anyway, I am happy it works now and maybe someone else looking for info on how to get Exim working with Sendgrid may find this useful. There isn't much information to be found for Exim/Sendgrid on DirectAdmin specifically.

PS. Don't forget that you also need to verify your domain on the Sendgrid website, either by adding the required CNAME records to the DNS zone of your domain, or by adding verified email addresses individually. If you forget this step your emails will be rejected with error:
SMTP error from remote mail server after end of data: 550 The from address does not match a verified Sender Identity. Mail cannot be sent until this error is resolved.
 
Last edited:
Interesting. Have you added and verified all the domains inside your SendGrid account as well?

They said I must add all domains that I host in DA to SG and verify those needed to use SG's email systems.

Then only emails from those domains will pass through.

And for me, that's not a Smarthost system.
 
Yes, I added the main domain on the Sendgrid website (there is only 1 primary domain on this server) and I created the CNAME records and verified them. Then I also added the 2 sub-domains "server. " and "mail. ", added their CNAME records and verified those as well. Then on top of that I added two individual email addresses diradmin@..... and root@.... so I will get those emails as well.

And I completely agree with you, this entire situation is shit. Both the fact that I need a third-party service just to send email, as well as the entire verification process that Sendgrid requires. I have another dedicated server in a datacenter which does not restrict port 25 and everything is setup and running correctly in less than an hour. This took me a week and a half in total, and it is completely unnecessary. I know of course why they block port 25, but they know me. I have an account with them, address info, credit card info, etc. It is not like I'm going to use the account to send spam and they don't know who I am.
Anyway, it's working now and I just want to forget this entire shit process.
2021 and we have to jump through stupid hoops like this. I won't call this 'progress'.
 
I tried GCP and wasted almost a few days to figure out that GCP doesn't open 25. Most providers will open based on request and manual verification. But GCP won't unless you pour $$$$ into their pocket each month and have a dedicated accounts manager.

One of the main reasons, I guess, is that they need to sell more G Suite accounts. Do you see the connection? More money to them, like a cross-sell of SEO courses =D

I tried and used many VPS providers, and all of the opened Port 25, except the Big G.

SendGrid and many other providers won't offer Smarthost. I spent hours with those settings and finally settled with https://www.smtp2go.com/setupguide/exim/.

This is the right way to do that. You can add any domains and email addresses in your DA, and no need to verify them in the SMTP2GO account individually.
 
Yeah, there were alternatives for Sendgrid. We have a similar account with AWS, but I wanted everything in GCP. Then when it turned out it wasn't possible, I stumbled on Sendgrid because it is specifically mentioned on the GCP website, among with a few others. But Sendgrid allows more emails in the free account for GCP users than you get if you get an account on Sendgrid's website itself. So I went with that.

I looked at the smtp2go link. As far as setup for Exim is concerned, it looks very similar to Sendgrid, almost identical. But maybe the setup that you need to do on their website is easier than Sendgrid.
 
I now ran into the next problem.
I tried to create a forwarder that would automatically forward an incoming mail for a specific email account to another external email address.
That does not work and results in this error: 550 The from address does not match a verified Sender Identity.
I actually understand where that error is coming from. The "from" address contains the email address of the original sender, not the forwarding mail address, and the domain of the original sender is not verified of course.
So I suspect that forwarding is yet another function that won't be possible. :(
 
We are using Google compute engine and I can confirm the settings here do work. Start to finish less than 30 minutes to log in, download the files, change these settings, do some other research (to make sure), and restart exim.

I've done this before using GCE and another panel, which uses a similar approach.
 
If you ever find a good solution for doing email 'forwarding' with GCE/Sendgrid, please let us know.
 
Tried this myself just now and based on a recent post I saw on another panel's forum, it looks like it's not possible.
 
Yeah, after a discussion with a support guy of Sendgrid, they confirmed this is not possible.

Another problem I ran in to: mail from admin@hostname.mydomain.com and root@hostname.mydomain.com was arriving correctly. But mail from diradmin@hostname.mydomain.com was not. I have no idea why. Both hostname.mydomain.com as well as those 3 email addresses individually are verified correctly on the Sendgrid website. Yet only admin and root work, but diradmin does not (results in 550 Sender not verified error).
I solved this by adding diradmin_envelope in directadmin.conf:

Version 1.62.0 | Directadmin Docs

DirectAdmin Knowledge Base
www.directadmin.com
 
You must log in or register to reply here.
Back
Top