Apache Down After Update to 2.4.59 - URGENT

R

Richpark

Verified User
Joined
Sep 25, 2014
Messages
55
I don't understand why this has happened, I've updated Apache so many times without issues. Yet another DA change?

Code: 
Apr 15 14:21:24 ***.co.uk systemd[1]: Starting The Apache HTTP Server...
Apr 15 14:21:24 ***.co.uk httpd[25251]: AH00526: Syntax error on line 136 of /etc/httpd/conf/extra/httpd-ssl.conf:
Apr 15 14:21:24 ***.co.uk httpd[25251]: SSLCACertificateFile: file '/etc/httpd/conf/ssl.crt/server.ca' does not exist or is empty
Apr 15 14:21:24 ***.co.uk systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE
Apr 15 14:21:24 ***.co.uk systemd[1]: Failed to start The Apache HTTP Server.
Apr 15 14:21:24 ***.co.uk systemd[1]: Unit httpd.service entered failed state.
Apr 15 14:21:24 ***.co.uk systemd[1]: httpd.service failed.

I have some identical servers where I haven't updated, the file "/etc/httpd/conf/ssl.crt/server.ca" still exists on those.

On the server I've updated, I see this:

Code: 
[/etc/httpd/conf/ssl.crt]> ls -la
total 20
drwxr-xr-x. 2 root root 4096 Apr 15 14:14 .
drwx--x---. 6 root root 4096 Apr 15 14:14 ..
-rw-------  1 root root  769 Apr 15 14:14 dhparams.pem
-rw-------  1 root root    0 Apr 15 14:14 server.ca
-rw-------  1 root root 1021 Apr 15 14:14 server.crt
-rw-------  1 root root 1021 Apr 15 13:48 server.crt.backup
-rw-------  1 root root    0 Apr 15 14:14 server.crt.combined

That shows server.ca has been updated today and blanked out. Why?!

I've rebuilt Apache twice and it's now blank every time. Please advise because I need this server up FAST.
 
My file says on line 136 :
#SSLCACertificateFile /etc/httpd/conf/ssl.crt/server.ca
Click to expand...

# Certificate Authority (CA):
# Set the CA certificate verification path where to find CA
# certificates for client authentication or alternatively one
# huge file containing all of them (file must be PEM encoded)
# Note: Inside SSLCACertificatePath you need hash symlinks
# to point to the certificate files. Use the provided
# Makefile to update the hash symlinks after changes.
#SSLCACertificateFile /etc/httpd/conf/ssl.crt/server.ca
#SSLCACertificatePath /etc/httpd/conf/ssl.crt
#SSLCACertificateFile /etc/httpd/conf/ssl.crt/ca-bundle.crt
Click to expand...

[root@srv1 ssl.crt]# ls -la
total 28
drwxr-xr-x 2 root root 4096 Apr 15 09:17 .
drwx--x--- 6 root root 4096 Apr 15 09:17 ..
-rw------- 1 root root 769 Apr 15 09:17 dhparams.pem
-rw------- 1 root root 1827 Apr 15 09:17 server.ca
-rw------- 1 root root 2114 Apr 15 09:17 server.crt
-rw------- 1 root root 2114 Feb 16 15:13 server.crt.backup
-rw------- 1 root root 3941 Apr 15 09:17 server.crt.combined
Click to expand...
 
The default DA file here:

/usr/local/directadmin/custombuild/configure/ap2/conf/extra/httpd-ssl.conf

Doesn't have that line commented out, so I don't know why yours is.

Regardless, I can see that your server.ca file has 1827 bytes of data, whereas mine has zero, which is the problem.
 
Richard G said:
Yes. New easier command now.

/usr/local/directadmin/scripts/letsencrypt.sh server_cert
Click to expand...
Here's the thing, I already have Let's Encrypt working on the server, I don't want to activate DA's implementation for fear it will mess up my own.

I rebuild Apache as recently as 2 months ago and I didn't have this problem. I just need to understand why DA is altering my SSL files when I rebuild Apache and why only the server.ca file is blank.

This has clearly worked perfectly for years, something has changed.
 
Richpark said:
Here's the thing, I already have Let's Encrypt working on the server, I don't want to activate DA's implementation for fear it will mess up my own.
Click to expand...
Well, I wouldn't know why you want to make life harder by using your own LE implementation while DA's implementation works fine. But oke, you are ofcourse entitled to your own choices.

The command I gave you is -only- for the hostname certificate, nothing more. So this will be for the hostname and things using hostname like Roundcube and Exim, so this would create your server.ca and will not interfere with any domain SSL's.

Maybe something changed because of the command change and DA tried to update your server certificate which might have failed due to this change.
But I'm not sure if that could indeed be the cause or the change which messed things up for you or something else is the reason.
 
Richard G said:
Well, I wouldn't know why you want to make life harder by using your own LE implementation while DA's implementation works fine. But oke, you are ofcourse entitled to your own choices.

The command I gave you is -only- for the hostname certificate, nothing more. So this will be for the hostname and things using hostname like Roundcube and Exim, so this would create your server.ca and will not interfere with any domain SSL's.

Maybe something changed because of the command change and DA tried to update your server certificate which might have failed due to this change.
But I'm not sure if that could indeed be the cause or the change which messed things up for you or something else is the reason.
Click to expand...
Because I implemented Let's Encrypt before DA even thought about it. I've been using DA since 2005.

Every time I build Apache, the files in "/etc/httpd/conf/ssl.crt" are updated. I never noticed this before because I never had a problem, I don't know if it's always been this way or not.

Regardless, the .crt file gets generated just fine, only the .ca file is blank. I see Eric above has commented out this line:

#SSLCACertificateFile /etc/httpd/conf/ssl.crt/server.ca

I can easily do the same, I just don't know if it will break anything else. If it's safe, I'll do that. This is just a self-signed certificate, I assume, so I don't know what the .ca file is doing.

The fact is, if server.ca can now be blank (which I think I've proved it can) then httpd-ssl.conf should have it commented out. Eric has clearly done this at some point for a reason.
 
Richpark said:
Because I implemented Let's Encrypt before DA even thought about it. I've been using DA since 2005.
Click to expand...
Oh that's a very good one! Nicely done then!! Compliments! (y) (y)

Richpark said:
I see Eric above has commented out this line:
Click to expand...
Well that's Eric's problem then. Because by default it's not commented out, there is no # in front of that file. I checked several servers and they all have it uncommented, so no # in front of it. At least here:
/usr/local/directadmin/custombuild/configure/ap2/conf/extra/httpd-ssl.conf

Seems @ericosman has something odd to investigate then. ;)

So no commenting out will not fix it as that is the location the httpd.conf is looking for anyway. Issue is the find out as to why your files got zero'd.
I hope you can find something in one of the DA logs which might explain this.

Since you're using DA that long I presume you're using a lifetime license, so no ticket support, unless a bug can be prooved.

Maybe @zEitEr has an idea when he has time to look at this topic.
 
Apache will start if I copy the server.ca from one of my other servers onto this one.

However, as soon as I build Apache, server.ca becomes blank again.

This is INSANE and has to be a newly introduced DA bug. Or does DA now require their own Let's Encrypt activated?

Even then, this doesn't make sense. For example, what if I purchased an SSL certificate for the server from DigiCert or similar and installed it myself? How can I live with DA overwriting it every time I build Apache?

Something is wrong here, building Apache shouldn't overwrite my SSL, full stop. Yes, I'm a lifetime licence holder, no I don't have support, but surely this is a bug and should be addressed. I can't be the only one who's going to run into this.

The only solution I can see is to chattr my SSL certificate files so that DA can't touch them.
 
Richard G said:
/usr/local/directadmin/custombuild/configure/ap2/conf/extra/httpd-ssl.conf
Click to expand...
# Certificate Authority (CA):
# Set the CA certificate verification path where to find CA
# certificates for client authentication or alternatively one
# huge file containing all of them (file must be PEM encoded)
# Note: Inside SSLCACertificatePath you need hash symlinks
# to point to the certificate files. Use the provided
# Makefile to update the hash symlinks after changes.
SSLCACertificateFile /etc/httpd/conf/ssl.crt/server.ca
#SSLCACertificatePath /etc/httpd/conf/ssl.crt
#SSLCACertificateFile /etc/httpd/conf/ssl.crt/ca-bundle.crt
Click to expand...
But in /etc/httpd/conf/ssl.crt/server.ca it's
# Certificate Authority (CA):
# Set the CA certificate verification path where to find CA
# certificates for client authentication or alternatively one
# huge file containing all of them (file must be PEM encoded)
# Note: Inside SSLCACertificatePath you need hash symlinks
# to point to the certificate files. Use the provided
# Makefile to update the hash symlinks after changes.
#SSLCACertificateFile /etc/httpd/conf/ssl.crt/server.ca
#SSLCACertificatePath /etc/httpd/conf/ssl.crt
#SSLCACertificateFile /etc/httpd/conf/ssl.crt/ca-bundle.crt
Click to expand...
 
Richpark said:
However, as soon as I build Apache, server.ca becomes blank again.
Click to expand...

While I don't have an idea on why you have the issue, but you might use a Directadmin hook and insert a content to the CA-file (as a bypass), which is actually an intermediate Let's Encrypt certificate. E.g.:

- /usr/local/directadmin/custombuild/custom/hooks/apache/post/update_ca_file.sh
You will need to create the folder /usr/local/directadmin/custombuild/custom/hooks/apache/post/ and a file update_ca_file.sh with a content of this kind:
Bash: 
#!/bin/bash
wget https://letsencrypt.org/certs/lets-encrypt-r3.pem -O /etc/httpd/conf/ssl.crt/server.ca
chmod 600 /etc/httpd/conf/ssl.crt/server.ca

The hook will work only after:

Code: 
./build apache

or

Code: 
da build apache

commands, it won't be triggered with any update commands.

See: https://docs.directadmin.com/developer/hooks/ for more details.
 
We encountered the same issue for 2 VMs - failed to start Apache 2.4.59
(Both AlmaLinux 9.3 and Apache 2.4.59)

----

After updating Apache 2.4.59
One is /etc/httpd/conf/ssl.crt/server.crt.combined becomes 0 filesize

We simply run
Code: 
cd /etc/httpd/conf/ssl.crt/
cat server.crt server.ca > server.crt.combined
service httpd restart  # OK to start

----

Another one is /etc/httpd/conf/ssl.crt/server.crt.combined is not updated
(i.e. server.crt.combined seems not matched with server.crt , server.key)

We also simply run
Code: 
cd /etc/httpd/conf/ssl.crt/
cat server.crt server.ca > server.crt.combined
service httpd restart  # OK to start
 
We have the same issue on several servers so far, I would like to know the root cause aswell in order to fix it.
 
You must log in or register to reply here.
Back
Top