MariaDB client socket in jailed environment

[sysdev]

When users have isolated php-fpm running, the jail get a mariadb socket. A mariadb restart afterwards creates a new socket with a new inode, but the jail seems to keep holding on to the old socket, resulting in weird db connection errors when visiting sites.

Normally (iirc) a jail stops about 10 minutes after the last web visitor. So low traffic sites get a fresh environment with a new socket from a new mariadb instance. Sites with a bit of decent traffic can have php masters processes running for months, even surviving php updates, and only fail if the socket changes.

If you do a filemount in the jail, you mount the inode making it point to nothing (hopefully) when mariadb restarts.

Mondays are always funny.

 

[fln]

@sysdev, the jailshell mounts a directory with the mariadb socket, not a socket inode. It is configured to work like this in the mariadb systemd service file and in the jailshell script. If you see an inode mount it is very likely you have customized one of them (service file or jail script). Removing the customizations will make sure mariadb restarts cause no problems for the long running jail shell processes.

 

[sysdev]

@sysdev, the jailshell mounts a directory with the mariadb socket, not a socket inode. It is configured to work like this in the mariadb systemd service file and in the jailshell script. If you see an inode mount it is very likely you have customized one of them (service file or jail script). Removing the customizations will make sure mariadb restarts cause no problems for the long running jail shell processes.

I have no customizations afaik. Running since aug-2025, all updates etc. But I do have a:

[root@popcorn my.cnf.d]# grep sock *
directadmin-socket.cnf:socket=/var/lib/mysql/mysql.sock
[root@popcorn my.cnf.d]# ll directadmin-socket.cnf
-rw-r--r-- 1 root root 42 Nov 12  2025 directadmin-socket.cnf

This creates a socket in /var/lib/mysql/mysql.sock

And in /usr/bin/jailshell I therefore end up in the first branch :

# Legacy MySQL socket mode
extra_args=""
if [ ! -L /var/lib/mysql/mysql.sock ]; then
        extra_args="--bind-try     /var/lib/mysql/mysql.sock     /run/mysql/mysql.sock"
else
        extra_args="--ro-bind-try  /run/mysql                    /run/mysql"
fi

and doing a file bind.

 

[fln]

@sysdev, this is a client configuration file, it does not affect where server creates the socket file:

# cat /etc/mysql/conf.d/directadmin-socket.cnf
[client]
socket=/var/lib/mysql/mysql.sock

On standard servers the socket is created in /run/mysql/mysql.sock because of:

# cat /etc/systemd/system/mysqld.service
...
ExecStartPre=-ln --symbolic --force --no-target-directory /run/mysql/mysql.sock /var/lib/mysql/mysql.sock
ExecStart=/usr/local/mysql/bin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION --datadir=/var/lib/mysql --socket=/run/mysql/mysql.sock
...

The old locations is just a symlink for backwards compatiblity:

# stat /var/lib/mysql/mysql.sock
  File: /var/lib/mysql/mysql.sock -> /run/mysql/mysql.sock

The extra_args="--ro-bind-try /run/mysql /run/mysql" code branch is executed in the jailshell script.

 

[sysdev]

@sysdev, this is a client configuration file, it does not affect where server creates the socket file:

# cat /etc/mysql/conf.d/directadmin-socket.cnf
[client]
socket=/var/lib/mysql/mysql.sock

On standard servers the socket is created in /run/mysql/mysql.sock because of:

# cat /etc/systemd/system/mysqld.service
...
ExecStartPre=-ln --symbolic --force --no-target-directory /run/mysql/mysql.sock /var/lib/mysql/mysql.sock
ExecStart=/usr/local/mysql/bin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION --datadir=/var/lib/mysql --socket=/run/mysql/mysql.sock
...

The old locations is just a symlink for backwards compatiblity:

# stat /var/lib/mysql/mysql.sock
  File: /var/lib/mysql/mysql.sock -> /run/mysql/mysql.sock

The extra_args="--ro-bind-try /run/mysql /run/mysql" code branch is executed in the jailshell script.

That looks different than I see overhere.

[root@popcorn ~]# cat /etc/systemd/system/mysqld.service|grep sock
ExecStart=/usr/local/mysql/bin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock
[root@popcorn ~]# ll /var/lib/mysql/mysql.sock
srwxrwxrwx 1 mysql mysql 0 Oct  9 18:08 /var/lib/mysql/mysql.sock
[root@popcorn ~]# file /var/lib/mysql/mysql.sock
/var/lib/mysql/mysql.sock: socket

My services file is dated Nov 12 2025 and hasn't been changed (by me).

Any idea?

 

[Ohm J]

@sysdev try re-build/update ?
The instruction in the past, DA recommended to re-build the database service together with PHP,
only old software that EOL not trigger to re-build by custombuild automatics. That's the problem, did DA enable isolated-fpm by default now ?

 

[sysdev]

@sysdev try re-build/update ?
The instruction in the past, DA recommended to re-build the database service together with PHP,
only old software that EOL not trigger to re-build by custombuild automatics. That's the problem, did DA enable isolated-fpm by default now ?

It's updated on every aspect, os, da, custombuild. Server is installed in Aug-2025.
I'll rebuild it tonight and see what happens.