Let's Encrypt renewals failing - complaining about CAA record

[jvdwilk]

On our production server we currently see that Let's Encrypt renewals are failing, due to an absent CAA record ?

[npnservers.com] acme: error: 403 :: POST :: https://acme-v02.api.letsencrypt.org/acme/finalize/901xxx94/450xxxx772 :: urn:ietf:params:acme:error:caa :: Error finalizing order :: Rechecking CAA for "smtp.npnservers.com" and 1 more identifiers failed. Refer to sub-problems for more information, problem: "urn:ietf:params:acme:error:caa" :: Error finalizing order :: While processing CAA for smtp.npnservers.com: DNS problem: SERVFAIL looking up CAA for smtp.npnservers.com - the domain's nameservers may be malfunctioning, problem: "urn:ietf:params:acme:error:caa" :: Error finalizing order :: While processing CAA for mail.npnservers.com: DNS problem: SERVFAIL looking up CAA for npnservers.com - the domain's nameservers may be malfunctioning, url:

Now, since we do NOT have CAA records enabled in the DirectAdmin settings, none of the domains on this server has (or ever had) a CAA record in the domain. It used to work fine, but now, since last week, it has been failing.

Is a CAA record going to be a hard requirement for requesting a Let's Encrypt certificate, or would the same error occure even if I had a CAA record present? I do see others reporting similar Let's Encrypt failures, WITH CAA records present. Could be related, or could not be?

Manually re-creating a new SSL certificate for this domain fails with the same error btw, so it's not just related to domain renewals or the automatic renewal script.

Anyone with ideas on how to troubleshoot, or work around this?

Thanks,
Jorge.

 

[jvdwilk]

This is too crazy for me...
More testing reveiled that it truely is some kind of DNS issue, but an intermittent one.
Using https://letsdebug.net/ and hitting 'retry' a couple of times first showed me that their was some kind of DNS error, but every time I hit 'retry' the error was pointing to something else. Sometimes a CAA record (which is not in the domain at all), sometimes the A record, sometimes the AAAA record. But also sometimes no error was found.

Then just re-creating the Let's Encrypt certificate for this domain, and retrying on failure: after 3 failures it said 'OK' and accepted the new SSL certificate.

Now... how to troubleshoot THIS ..??
<sigh>

 

[Dannik]

I experience the same problems. I have 2 servers, one running LE 1.x and 1 LE 2.x. The second server is kind of a testing server before I run all updates to the first (production) server. But on both servers the same problems occur, so obviously is has ntohing to do with the LE version...?

Some of the errors I get:

Processing authorization for somedomain.nl...
Challenge is valid.
Processing https://acme-v02.api.letsencrypt.org/acme/authz-v3/6494934662...
Nonce is empty. Exiting. dig output of acme-v02.api.letsencrypt.org:
prod.api.letsencrypt.org.
ca80a1adb12a4fbdac5ffcbc944e9a61.pacloudflare.com.
172.65.32.248
Full nonce request output:

2020/08/20 00:29:36 [INFO] [somedomain.net, mail.somedomain.net, pop.somedomain.net, smtp.somedomain.net, www.somedomain.net] acme: Validations succeeded; requesting certificates
2020/08/20 00:29:40 Could not obtain certificates:
error: one or more domains had a problem:
[mail.somedomain.net] acme: error: 403 :: POST :: https://acme-v02.api.letsencrypt.org/acme/finalize/94078321/4768680158 :: urn:ietf: params:acme:error:caa :: Error finalizing order :: While processing CAA for mail.somedomain.net: DNS problem: SERVFAIL looking up CAA for mail.somedomain.net - the domain's nameservers may be malfunctioning, etc...

After digging for a while it seems you can/must enable a certain setting in directadmin.conf: dns_caa=1 and restart DirectAdmin. After changing this my testserver gave me 1 more error after manually requesting a certificate, but the second request was without errors. Testing it on my production server (and a wildcard certificate with mutiple domains/pointers) succeeded also.

I'll be watching this though...

 

[youds]

I have to say, I’m not in production but this letsencrypt script does seem to have connectivity or gotcha issues. It’s the only *real* problem I have faced from stock panel and skin.

Yes it’s free, but only if it works. I’ve found wildcard is a help.

 

[jvdwilk]

Well... to be fair I have to mention now that after analysing all the errors, everything was pointing to DNS issues.
And while all DNS servers where working, I did find that one DNS server had intermittant issues with connectivity over IPv6.
After addressing these IPv6 connectivity issues on the DNS server, the SERVFAIL from Let's Encrypt has - sofar - not returned anymore.

Not very resilient, for why not just checking another DNS server in case of a temporary error, but the issue seems indeed to have been a DNS server issue...