Let's Encrypt stopped working

[ZipperZapper]

A few weeks ago I upgraded my server from Ubuntu 18.04 to Ubuntu 20.04. I did the release upgrade on the machine itself and rebuild all the Directadmin software afterwards with the Custombuild plugin.

All seemed to work fine, but now a few weeks later it turns out Let's Encrypt somehow stopped working. Every domain on my server gives the same error sequence:

2023:03:02-16:36:50: LetsEncrypt(2724347): exit code: 1 for domain='XXX.XX' : 2023/03/02 16:34:45 [INFO] [XXX.XX] acme: Obtaining SAN certificate
2023/03/02 16:34:46 [INFO] [XXX.XX] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/XXXXX
2023/03/02 16:34:46 [INFO] [www.XXX.XX] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/XXXXX
2023/03/02 16:34:46 [INFO] [XXX.XX] acme: Could not find solver for: tls-alpn-01
2023/03/02 16:34:46 [INFO] [XXX.XX] acme: use http-01 solver
2023/03/02 16:34:46 [INFO] [www.XXX.XX] acme: Could not find solver for: tls-alpn-01
2023/03/02 16:34:46 [INFO] [www.XXX.XX] acme: use http-01 solver
2023/03/02 16:34:46 [INFO] [XXX.XX] acme: Trying to solve HTTP-01
2023/03/02 16:35:38 [INFO] [www.XXX.XX] acme: Trying to solve HTTP-01
2023/03/02 16:36:49 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/XXXXX
2023/03/02 16:36:49 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/XXXXX
2023/03/02 16:36:50 Could not obtain certificates:
    error: one or more domains had a problem:
[XXX.XX] acme: error: 400 :: urn:ietf:params:acme:error:connection :: XXX.XXX.XXX.XXX: Fetching https://XXX.XX/.well-known/acme-challenge/XXXXXXX: Timeout during connect (likely firewall problem)
[www.XXX.XX] acme: error: 400 :: urn:ietf:params:acme:error:connection :: XXX.XXX.XXX.XXX: Fetching https://www.XXX.XX/.well-known/acme-challenge/XXXXXXXX: Timeout during connect (likely firewall problem)
Certificate generation failed.

I use the ConfigServer Security & Firewall and did not change any paramaters after the Ubuntu upgrade. I also tried to disable it completely because of the firewall reference in the log, but it does not make a difference.

I tried rebuilding all the Directadmin software again:

cd /usr/local/directadmin/custombuild
./build update
./build all

But again, no difference and the same error messages.
I searched in this forum and general Google, but I can't seem to find the right way to debug this.

Is there anybody here who knows this error and knows how to debug this?

 

[Richard G]

Most likely this is a firewall issue or some DNS hostname configuration not correct since it's happening to all domains.
Try this:
nmap domain.com -Pn
ofcourse replace domain.com with a domain name having the issues. If possible not from the server having the issues but from another place.

Post the output here.

 

[ZipperZapper]

Nothing special to see there. If something was seriously wrong on the firewall/DNS front I would say the websites itself would be unreachable as well, which is not the case.

Starting Nmap 7.80 ( https://nmap.org ) at 2023-03-02 18:04 CET
Nmap scan report for XXX.XX (XXX.XXX.XXX.XXX)
Host is up (0.00026s latency).
Other addresses for XXX.XX (not scanned): XXXIPv6XXX
rDNS record for XXX.XXX.XXX.XXX: X.XXX.XX
Not shown: 991 closed ports
PORT     STATE SERVICE
80/tcp   open  http
443/tcp  open  https

Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds

 

[Richard G]

Hmmz..... I presume these ports are both open incombing but also outgoing for both ipv4 and ipv6.
Hopefully it's not some block for LE ip adresses somewhere.

Hard do find out more without additional information like ip and hostname. Or you can ask over at the LE support forums. But they will also need additional info.

 

[ZipperZapper]

I tried generating a certificate with ZeroSSL which is apparently also an option within Directadmin these days (didn't know that before) and that one is succesful. Strange.

Doesn't really fix the problem but it's at least an escape for some certs that where close to expiring now.