Can't update MySQL/MariaDB with Custombuild on Debian 9, can't connect through socket

HolyDiver

Verified User
Joined
Feb 21, 2016
Messages
51
Code:
~# /usr/local/directadmin/custombuild/build mysql
mv: 'mariadb-5.5.58-linux-x86_64.tar.gz' and '/usr/local/mariadb-5.5.58-linux-x86_64.tar.gz' are the same file
Stopping mysqld ...
Synchronizing state of mysqld.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable mysqld
update-rc.d: error: mysqld Default-Start contains no runlevels, aborting.
Ensuring local-infile is disabled for security reasons in MySQL configuration file...
Giving mysqld a few seconds to start up...
/usr/local/mysql/bin/mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/usr/local/mysql/data/mysql.sock' (2) when trying to connect
FATAL ERROR: Upgrade failed
Restarting MySQL.

I don't understand what's wrong with the socket. Every other application including mysqld has MySQL/MariaDB working perfectly with that same socket. It's only failing during upgrades like these.
I've looked through past posts on this forum and tried suggestions, but haven't found a solution yet.

Code:
~# ls -l /usr/local/mysql/data/mysql.sock
srwxrwxrwx 1 mysql mysql 0 Jan 12 18:34 /usr/local/mysql/data/mysql.sock

Code:
~# nano /etc/my.cnf
[client]
port                    = 3306
socket                  = /usr/local/mysql/data/mysql.sock

[mysqld]
local-infile = 0
interactive_timeout     = 60
join_buffer_size        = 128K
key_buffer_size         = 100M
max_allowed_packet      = 16M
max_connections         = 40
max_heap_table_size     = 16M
myisam_sort_buffer_size = 64M
port                    = 3306
query_cache_size        = 4M
socket                  = /usr/local/mysql/data/mysql.sock
table_open_cache        = 512
thread_cache_size       = 8
tmp_table_size          = 16M
wait_timeout            = 30

[mysqldump]
max_allowed_packet      = 16M
quick

[mysql]
no-auto-rehash

[myisamchk]
key_buffer_size         = 64M
read_buffer             = 2M
sort_buffer_size        = 64M
write_buffer            = 2M

[mysqlhotcopy]
interactive-timeout

Code:
~# /usr/local/directadmin/conf/mysql.conf
user=da_admin
passwd=**********
socket=/usr/local/mysql/data/mysql.sock

Code:
~# /usr/local/directadmin/conf/my.cnf
[client]
user=da_admin
password=**********
socket=/usr/local/mysql/data/mysql.sock

Code:
~# mysql --help --verbose
mysql  Ver 15.1 Distrib 5.5.58-MariaDB, for Linux (x86_64) using readline 5.1
Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.

Usage: mysql [OPTIONS] [database]

Default options are read from the following files in the given order:
/etc/my.cnf /etc/mysql/my.cnf ~/.my.cnf
The following groups are read: mysql client client-server client-mariadb

Code:
~# nano /home/mysql/**********.com.err
180112 16:30:07 [Note] /usr/local/mysql/bin/mysqld: Normal shutdown

180112 16:30:07 [Note] Event Scheduler: Purging the queue. 0 events
180112 16:30:07  InnoDB: Starting shutdown...
180112 16:30:09  InnoDB: Shutdown completed; log sequence number 234248205
180112 16:30:10 [Note] /usr/local/mysql/bin/mysqld: Shutdown complete

180112 16:30:10 mysqld_safe mysqld from pid file /usr/local/mysql/data/mysqld.pid ended
180112 16:31:31 mysqld_safe Starting mysqld daemon with databases from /usr/local/mysql/data
180112 16:31:34 [Note] /usr/local/mysql/bin/mysqld (mysqld 5.5.58-MariaDB) starting as process 18726 ...
180112 16:31:35 InnoDB: The InnoDB memory heap is disabled
180112 16:31:35 InnoDB: Mutexes and rw_locks use GCC atomic builtins
180112 16:31:35 InnoDB: Compressed tables use zlib 1.2.3
180112 16:31:35 InnoDB: Using Linux native AIO
180112 16:31:35 InnoDB: Initializing buffer pool, size = 128.0M
180112 16:31:35 InnoDB: Completed initialization of buffer pool
180112 16:31:35 InnoDB: highest supported file format is Barracuda.
180112 16:31:42  InnoDB: Waiting for the background threads to start
180112 16:31:43 Percona XtraDB (http://www.percona.com) 5.5.55-MariaDB-38.9 started; log sequence number 234248205
180112 16:31:43 [Note] Server socket created on IP: '0.0.0.0'.
180112 16:31:43 [Note] /usr/local/mysql/bin/mysqld: Normal shutdown

180112 16:31:43 [Note] Event Scheduler: Loaded 0 events
180112 16:31:43 [Note] /usr/local/mysql/bin/mysqld: ready for connections.
Version: '5.5.58-MariaDB'  socket: '/usr/local/mysql/data/mysql.sock'  port: 3306  Source distribution
180112 16:31:43 [Note] Event Scheduler: Purging the queue. 0 events
180112 16:31:43  InnoDB: Starting shutdown...
180112 16:31:52  InnoDB: Shutdown completed; log sequence number 234248205
180112 16:31:52 [Note] /usr/local/mysql/bin/mysqld: Shutdown complete

180112 16:31:53 mysqld_safe mysqld from pid file /usr/local/mysql/data/mysqld.pid ended
180112 16:31:53 mysqld_safe Starting mysqld daemon with databases from /usr/local/mysql/data
180112 16:31:53 [Note] /usr/local/mysql/bin/mysqld (mysqld 5.5.58-MariaDB) starting as process 19716 ...
180112 16:31:53 InnoDB: The InnoDB memory heap is disabled
180112 16:31:53 InnoDB: Mutexes and rw_locks use GCC atomic builtins
180112 16:31:53 InnoDB: Compressed tables use zlib 1.2.3
180112 16:31:53 InnoDB: Using Linux native AIO
180112 16:31:53 InnoDB: Initializing buffer pool, size = 128.0M
180112 16:31:53 InnoDB: Completed initialization of buffer pool
180112 16:31:53 InnoDB: highest supported file format is Barracuda.
180112 16:31:53  InnoDB: Waiting for the background threads to start
180112 16:31:54 Percona XtraDB (http://www.percona.com) 5.5.55-MariaDB-38.9 started; log sequence number 234248205
180112 16:31:54 [Note] Server socket created on IP: '0.0.0.0'.
180112 16:31:54 [Note] Event Scheduler: Loaded 0 events
180112 16:31:54 [Note] /usr/local/mysql/bin/mysqld: ready for connections.
Version: '5.5.58-MariaDB'  socket: '/usr/local/mysql/data/mysql.sock'  port: 3306  Source distribution
 
Last edited:
Hello,

Try and put the socket definition under every section of my.cnf file:

Code:
socket                  = /usr/local/mysql/data/mysql.sock

One of our servers with Debian has this:

Code:
[mysqld_safe]
port                           = 3306
socket                         = /usr/local/mysql/data/mysql.sock


[mysql_upgrade]
port                           = 3306
socket                         = /usr/local/mysql/data/mysql.sock


[mysqladmin]
port                           = 3306
socket                         = /usr/local/mysql/data/mysql.sock


[mysqlbinlog]
port                           = 3306
socket                         = /usr/local/mysql/data/mysql.sock


[mysqlcheck]
port                           = 3306
socket                         = /usr/local/mysql/data/mysql.sock


[mysqldump]
port                           = 3306
socket                         = /usr/local/mysql/data/mysql.sock


[mysqlimport]
port                           = 3306
socket                         = /usr/local/mysql/data/mysql.sock

since Mar 26 2017 without modifications.
 
I've added the socket to each section of my.cnf, but the same error keeps happening

Code:
~# nano /etc/my.cnf
[client]
port                    = 3306
socket                  = /usr/local/mysql/data/mysql.sock

[myisamchk]
key_buffer_size         = 64M
port                    = 3306
read_buffer             = 2M
socket                  = /usr/local/mysql/data/mysql.sock
sort_buffer_size        = 64M
write_buffer            = 2M

[mysql]
no-auto-rehash
port                    = 3306
socket                  = /usr/local/mysql/data/mysql.sock

[mysql_upgrade]
port                    = 3306
socket                  = /usr/local/mysql/data/mysql.sock

[mysqladmin]
port                    = 3306
socket                  = /usr/local/mysql/data/mysql.sock

[mysqlbinlog]
port                    = 3306
socket                  = /usr/local/mysql/data/mysql.sock

[mysqlcheck]
port                    = 3306
socket                  = /usr/local/mysql/data/mysql.sock

[mysqld]
interactive_timeout     = 60
join_buffer_size        = 128K
key_buffer_size         = 100M
local-infile            = 0
max_allowed_packet      = 16M
max_connections         = 40
max_heap_table_size     = 16M
myisam_sort_buffer_size = 64M
port                    = 3306
query_cache_size        = 4M
socket                  = /usr/local/mysql/data/mysql.sock
table_open_cache        = 512
thread_cache_size       = 8
tmp_table_size          = 16M
wait_timeout            = 30

[mysqld_safe]
port                    = 3306
socket                  = /usr/local/mysql/data/mysql.sock

[mysqldump]
max_allowed_packet      = 16M
port                    = 3306
quick
socket                  = /usr/local/mysql/data/mysql.sock

[mysqlhotcopy]
interactive-timeout
port                    = 3306
socket                  = /usr/local/mysql/data/mysql.sock

[mysqlimport]
port                    = 3306
socket                  = /usr/local/mysql/data/mysql.sock

Code:
~# /usr/local/directadmin/custombuild/build mysql
mv: 'mariadb-5.5.58-linux-x86_64.tar.gz' and '/usr/local/mariadb-5.5.58-linux-x86_64.tar.gz' are the same file
Stopping mysqld ...
Synchronizing state of mysqld.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable mysqld
update-rc.d: error: mysqld Default-Start contains no runlevels, aborting.
mv: cannot stat '/etc/mysql/my.cnf': No such file or directory
Ensuring local-infile is disabled for security reasons in MySQL configuration file...
Giving mysqld a few seconds to start up...
/usr/local/mysql/bin/mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/usr/local/mysql/data/mysql.sock' (2) when trying to connect
FATAL ERROR: Upgrade failed
Restarting MySQL.

Code:
~# nano /home/mysql/********.com.err
180112 22:46:09 mysqld_safe Starting mysqld daemon with databases from /usr/local/mysql/data
180112 22:46:11 [Note] /usr/local/mysql/bin/mysqld (mysqld 5.5.58-MariaDB) starting as process 3329 ...
180112 22:46:12 InnoDB: The InnoDB memory heap is disabled
180112 22:46:12 InnoDB: Mutexes and rw_locks use GCC atomic builtins
180112 22:46:12 InnoDB: Compressed tables use zlib 1.2.3
180112 22:46:12 InnoDB: Using Linux native AIO
180112 22:46:12 InnoDB: Initializing buffer pool, size = 128.0M
180112 22:46:13 InnoDB: Completed initialization of buffer pool
180112 22:46:13 InnoDB: highest supported file format is Barracuda.
180112 22:46:18  InnoDB: Waiting for the background threads to start
180112 22:46:19 Percona XtraDB (http://www.percona.com) 5.5.55-MariaDB-38.9 started; log sequence number$
180112 22:46:19 [Note] Server socket created on IP: '0.0.0.0'.
180112 22:46:19 [Note] /usr/local/mysql/bin/mysqld: Normal shutdown

180112 22:46:20 [Note] Event Scheduler: Loaded 0 events
180112 22:46:20 [Note] /usr/local/mysql/bin/mysqld: ready for connections.
Version: '5.5.58-MariaDB'  socket: '/usr/local/mysql/data/mysql.sock'  port: 3306  Source distribution
180112 22:46:20 [Note] Event Scheduler: Purging the queue. 0 events
180112 22:46:20  InnoDB: Starting shutdown...
180112 22:46:45  InnoDB: Shutdown completed; log sequence number 234248235
180112 22:46:45 [Note] /usr/local/mysql/bin/mysqld: Shutdown complete

180112 22:46:45 mysqld_safe mysqld from pid file /usr/local/mysql/data/mysqld.pid ended
180112 22:46:45 mysqld_safe Starting mysqld daemon with databases from /usr/local/mysql/data
180112 22:46:45 [Note] /usr/local/mysql/bin/mysqld (mysqld 5.5.58-MariaDB) starting as process 4459 ...
180112 22:46:45 InnoDB: The InnoDB memory heap is disabled
180112 22:46:45 InnoDB: Mutexes and rw_locks use GCC atomic builtins
180112 22:46:45 InnoDB: Compressed tables use zlib 1.2.3
180112 22:46:45 InnoDB: Using Linux native AIO
180112 22:46:45 InnoDB: Initializing buffer pool, size = 128.0M
180112 22:46:45 InnoDB: Completed initialization of buffer pool
180112 22:46:45 InnoDB: highest supported file format is Barracuda.
180112 22:46:45  InnoDB: Waiting for the background threads to start
180112 22:46:46 Percona XtraDB (http://www.percona.com) 5.5.55-MariaDB-38.9 started; log sequence number$
180112 22:46:46 [Note] Server socket created on IP: '0.0.0.0'.
180112 22:46:46 [Note] Event Scheduler: Loaded 0 events
180112 22:46:46 [Note] /usr/local/mysql/bin/mysqld: ready for connections.
Version: '5.5.58-MariaDB'  socket: '/usr/local/mysql/data/mysql.sock'  port: 3306  Source distribution
 
Seems something is really wrong with your setup. First you had issues with PHP-FPM sockets, and now you have an issue with MySQL. Did you by any chance change recursively permissions on root and/or parent directories?

Code:
ls -la /usr/local/mysql/data/mysql.sock
ls -lda /usr/local/mysql/data/
ls -lda /usr/local/mysql/
ls -lda /usr/local/
ls -lda /usr/
ls -lda /
?

Try to simlink /etc/my.cnf to /etc/mysql/my.cnf

Does OS in Directadmin license match the version you really run? I mean do you really have a directadmin license for Debian 9?

Do you have a native kernel? Or hardened from a 3rd party?

If nothing helps, you might need to contact a directadmin support for a free help (they might login to your server and check what is wrong)? Or feel free to hire me for this job.
 
What are you running at the moment? MySQL or MariaDB.
Are you still on custombuild 1?
 
Seems something is really wrong with your setup. First you had issues with PHP-FPM sockets, and now you have an issue with MySQL. Did you by any chance change recursively permissions on root and/or parent directories?
I think I have this issue with MySQL/MariaDB since I upgraded to Debian 8 last year, I used to think the upgrade sworked because the version number gets increased even with failed upgrades. I haven't touched the permission of those directories.
Code:
~# ls -lda /usr/local/mysql/data/mysql.sock
srwxrwxrwx 1 mysql mysql 0 Jan 12 22:46 /usr/local/mysql/data/mysql.sock

~# ls -lda /usr/local/mysql/data/
drwx--x--x 11 mysql mysql 4096 Jan 12 22:56 /usr/local/mysql/data/

~# ls -lda /usr/local/mysql/
drwxr-sr-x 12 mysql mysql 4096 Jan 12 22:46 /usr/local/mysql/

~# ls -lda /usr/local/
drwxrwsr-x 20 root staff 4096 Jan 12 22:46 /usr/local/

~# ls -lda /usr/
drwxr-xr-x 19 root root 4096 Jan  1 03:16 /usr/

~# ls -lda /
drwxr-xr-x 24 root root 4096 Jan  2 21:36 /

Try to simlink /etc/my.cnf to /etc/mysql/my.cnf
I've tried this, but the result is the same.

Does OS in Directadmin license match the version you really run? I mean do you really have a directadmin license for Debian 9?
Yes, the license is set on Debian 9 64-bit. I even recompiled DirectAdmin to be sure it matches.

Do you have a native kernel? Or hardened from a 3rd party?
It runs on the latest native kernel provided by apt-get.
Code:
~# uname -rv
4.9.0-5-amd64 #1 SMP Debian 4.9.65-3+deb9u2 (2018-01-04)

If nothing helps, you might need to contact a directadmin support for a free help (they might login to your server and check what is wrong)? Or feel free to hire me for this job.
Good to know, though I hope to learn from this :)

What are you running at the moment? MySQL or MariaDB.
Are you still on custombuild 1?
I'm running MariaDB 5.5 on Custombuild2.
 
Last edited:
Then to update you need to use this:

Code:
cd /usr/local/directadmin/custombuild
./build set mariadb 5.5
./build set mysql_inst mariadb
./build set mysql_backup yes
./build update
./build mariadb
 
Well permissions look more or less the same as I have here. The difference is only here:

Code:
drwxr-xr-x 81 mysql mysql 4096 Dec 27 05:08 /usr/local/mysql/data/

MySQL logs do not show errors.

If upgrade fails, I'd try and do a completely new installation of MariaDB on the server. Check https://help.directadmin.com/ for the instructions on how to re-install MySQL/MariaDB.
 
I tried Awd's suggestion to try ./build mariadb instead (I thought it'd just run ./build mysql in the background) and the actual first time it didn't gave the socket error, instead it said to use --force to force an upgrade (which should be the default setting according to the DA documentation). Every other attempt after that, including trying /usr/local/mysql/bin/mysql_upgrade --force --defaults-extra-file=/usr/local/directadmin/conf/my.cnf, gave the socket error again.

Changing the permissions of /usr/local/mysql/data/ to 755 had no effect either.

I'll be trying a new installation of MariaDB later today :)
 
An update about my situation for whoever googles this topic.

Didn't do a new installation, I couldn't afford any downtime due to a new business contract. But the issue solved itself. I didn't upgrade MariaDB for months, I didn't change any related config file. Keeping my Debian installment up-to-date seemed to have done the trick.
 
Ah, you're right! So it was fixed half a year ago already without me knowing :p
 
The problem returned with the latest 5.5.64 update for MariaDB. I changed nothing to my configuration since the successful 5.5.63 update two weeks ago. I did update from Debian 9.8 to Debian 9.9 this week.

Code:
Executing /usr/local/directadmin/plugins/custombuild/admin/build mysql..
Dumping database da_roundcube 
	Dumping database [...] 
	Dumping database mysql 
	Dumping database test 
--2019-05-01 16:54:38--  http://files6.directadmin.com/services/all/mariadb/5.5/5.5.64/mariadb-5.5.64-linux-glibc_214-x86_64.tar.gz
Resolving files6.directadmin.com (files6.directadmin.com)... 2001:16e8:1:c:62:148:174:106, 62.148.174.106
Connecting to files6.directadmin.com (files6.directadmin.com)|2001:16e8:1:c:62:148:174:106|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 305926101 (292M) [application/x-gzip]
Saving to: 'mariadb-5.5.64-linux-glibc_214-x86_64.tar.gz'


     0K .......... .......... .......... .......... ..........  0% 10.9M 27s
     [...]
     298750K .....                                                 100% 60.5M=11s

mv: 'mariadb-5.5.64-linux-glibc_214-x86_64.tar.gz' and '/usr/local/mariadb-5.5.64-linux-glibc_214-x86_64.tar.gz' are the same file
tar: write error
Stopping mysqld ...
Synchronizing state of mysqld.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable mysqld
update-rc.d: error: mysqld Default-Start contains no runlevels, aborting.
Ensuring local-infile is disabled for security reasons in MySQL configuration file...
Giving mysqld a few seconds to start up...
/usr/local/mysql/bin/mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/usr/local/mysql/data/mysql.sock' (2) when trying to connect
FATAL ERROR: Upgrade failed
Restarting MySQL.
Done!
 
Last edited:
Aaaaaaaand the latest 5.5.65 update fixed it again. Also seemed to have fixed an issue where the db connection is null sometimes.
 
Back
Top