All Sites down port 80 and 443 -error: Can't connect to ssl| SSL Socket write error | Broken Pipe | error:00000001:lib(0):func(0):reason(1)

C

codes9

Verified User
Joined
Sep 5, 2019
Messages
73
My server was rebooted by our VPS host. After this DirectAdmin has come back up but none of our sites are available:
  • Running Centos8 + Openlitespeed
  • I can login to Directadmin on port 2222 and I have SSH access.
  • All other http & https traffic seems down

What I've done so far:
  • I restarted httpd and checked logs. Did anything that pointed to an http issue
  • I first noticed symptoms of directadmin's PID file not being written. I've followed: https://forum.directadmin.com/threads/directadmin-service-never-wrote-its-pid-file.57435/
  • I've rebuilt all binaries with Custombuild as a result of some of the ssl errors after following this thread. After much troubleshooting and running DirectAdmin in debug mode the PID error disappeared.

The SSL errors persisit and look like the root cause. Sites are still down.

$ nano /var/log/directadmin/error.log
Code: 
2020:05:14-13:45:02: Can't connect to ssl!
2020:05:14-13:45:02: -> A failure in the SSL library occurred, usually a protocol error.
2020:05:14-13:45:02: -> error:00000001:lib(0):func(0):reason(1)
2020:05:14-14:22:22: Can't connect to ssl!
2020:05:14-14:22:22: ->syscall
2020:05:15-12:49:38: Can't connect to ssl!
2020:05:15-12:49:38: ->syscall
2020:05:15-12:49:43: SSL Socket write error (ret=-1): Some I/O error occurred.  The OpenSSL error queue may contain more information on the error.  I$
openssl error queue:
empty error queue.  ret=-1
errno: Broken pipe

2020:05:15-12:55:00: Can't connect to ssl!
2020:05:15-12:55:00: ->syscall
2020:05:15-15:41:29: Can't connect to ssl!
2020:05:15-15:41:29: ->syscall

Checked the hostname certificate for this service from my machine (I've replaced my hostname with example.com in the text below):

Code: 
openssl s_client -servername example.com -host example.com -port 443 | grep 'Verify return code'

write:errno=104

Verify return code: 0 (ok)

Found a post on this error code https://superuser.com/questions/297889/does-openssl-errno-104-mean-that-sslv2-is-disabled

At least on Linux, 104 is ECONNRESET for "Connection reset by peer" – in other words, the connection was forcibly closed with a TCP RST packet, either sent out by the server or spoofed by an intermediary.

I would try Wireshark/tshark on the Ubuntu server to see what actually gets sent. If the RST is real, it could be that the httpd process died – check the log files and dmesg just in case.
Click to expand...
  • I've tried rebuilding the hostname cert but found letsencrypt. However letsencrypt can't navigate to it's webserver to validate a new SSL cert.
  • Qualasys obviously is no help here Anyone have ideas where else I could lookeither since the webserver can't serve a page as a result of the error.
  • Tried disabling ssl for a domain to see if I can get to a domain without SSL. No access to this domain.
Navigating to the servers hostname:80 usually shows the status of the webserver. But this page is also down.
Firefox Reports the following when I navigate to any http or https page:

Code: 
Secure Connection Failed

An error occurred during a connection to www.atservice.co.za. PR_CONNECT_RESET_ERROR

    The page you are trying to view cannot be shown because the authenticity of the received data could not be verified.
    Please contact the website owners to inform them of this problem.

I've rebuilt Openlitespeed and OpenSSH

I'd really appreciate if anyone has ideas where I could dig for clues on this? I've been at this for a few hours and don't know where else to look?
 
Did you run
Code: 
cd /usr/local/directadmin/custombuild
./build update
./build rewrite_confs
 
Is this a new server?
was this the first reboot ever?
what has changed since the last reboot?
dnf all up to date?

Have you ever run
Code: 
./build all
 
Hi Brent :) Nice to see you again

It is a relatively new server. I have run "./build all" since installing. But we're on the same track.

I've just run a build all instruction before your messages came in.
I'll let you know what the outcome was shortly :coffee:
 
Hi Brent

Awesome instincts! Building everything again did solve the problem.

I'm unsure why this happened in the first place. Since the logs did not tell much.
But it seems resolved for now.

I'll be making a note of this. For future reference.

Codes9
 
Hmm, Don't recall changing any options on DirectAdmin prior to this happening.
Maybe on 8 May which was the last Admin entry. Sure I built after that :confused:

Thanks for the heads-up 😎. I'll be sure build each time after new options are selected. 😅
 
You must log in or register to reply here.
Back
Top