Build MariaDB deletes all databases

Z

ZipperZapper

Verified User
Joined
Nov 30, 2015
Messages
142
I don't know if this behaviour is intended or a bug, but for me it was a surprise.
I had some issues with MariaDB crashing the last week so I decided to rebuild it to see if it would fix things, wouldn't hurt either right?

Custombuild > Build Software > Build MariaDB.

I thought this worked fine and went on with my day, until some monitor alerts came up. Turns out all database users and databases itself were deleted in this process.
Luckily I have a good backup strategy and was able to restore the databases within half an hour, but it's not fun to do.

Is this intended behaviour? Maybe some sort of warning would be in place in that case.

Custombuild 2.0.0 (rev: 2895)
Directadmin 1.641
Ubuntu 18.04.6
 
Fs you say - mysql was crached, that's why it drops DBs.
You must restore mysql after crash, then make backups, check their consistency, and only after all this - rebuild/reinstall mysql if needed.
 
Crash was maybe a bit much. I had lot of "the service mysql stopped" messages all day, that's why I wanted to try a rebuild.
It was running, working fine and all data was there before I did the rebuild.
 
strange, and what was in logs about this "mysql stopped"? maybe openfiles limit or OOM killer?
 
I will open another topic about that.

This one is purely to know if Build MariaDB is intended to delete everything or not.
 
No, mariadb in normal cases didn't remove anything. We always update/upgrade Mariadb without data loss. Something went wrong in this specific case.
 
Yes, update/upgrade is never a problem in regular usage.

Mind this is about the "Build Software" tab in Custombuild, not the regular update procedure.
 
I think it just updated the datadir :) Your data is likely in /home/mysql. Am I right? Check:
Code: 
ls /var/lib/mysql
ls /home/mysql
 
Hi,

Yes, that seems to be the case. The db's in the var folder are all created during the restore this morning.
The home ones seem te be created when the websites were created.

Which folder is going to be backed up by the Admin Backup function? Should I delete one of the 2?
 
Please check if the following solves your problem:
Code: 
service mysqld stop
mv /var/lib/mysql /var/lib/mysql_backup
ln -s /home/mysql /var/lib/mysql
service mysqld start
 
Had the same issue yesterday on Debian, creating the

Code: 
ln -s /home/mysql /var/lib/mysql

symlink indeed fixed it for me. And indeed the databases are not deleted, just the datadir changed after upgrading without the symlink.
 
As I understand it correctly this will put the 'original' data back.

Is it needed? It's been a few hours since the restore and I don't want to cut another few hours of data out if it's not needed.

Edit: I mean: is it needed to make the symlink to make sure everything like backups and stuff keep working, or is it fine like this and will all systems use the new /var location by default.

Doing the symlink would have fixed it when the issue was happening this morning but I couldn't wait for the solution haha.
 
Better check with smtalk. As I understand it, if everything is working fine now, you can leave it as it is and remove the directory with the data you're not using anymore.

@smtalk I think the issue is that when executing ./build mariadb, Custombuild checks if /var/lib/mysql already exists, and when you're using /home/mysql as datadir (which a lot of our servers, with a default DA install, are using), /var/lib/mysql will not be found and a new MariaDB datadir will be created and configured here. Perhaps the Custombuild script could be tweaked, because I think a lot of users will experience this issue in the future when trying to update MariaDB.
 
wtptrs said:
Better check with smtalk. As I understand it, if everything is working fine now, you can leave it as it is and remove the directory with the data you're not using anymore.

@smtalk I think the issue is that when executing ./build mariadb, Custombuild checks if /var/lib/mysql already exists, and when you're using /home/mysql as datadir (which a lot of our servers, with a default DA install, are using), /var/lib/mysql will not be found and a new MariaDB datadir will be created and configured here. Perhaps the Custombuild script could be tweaked, because I think a lot of users will experience this issue in the future when trying to update MariaDB.
Click to expand...
It's been tweaked already in the latest revision.
 
wtptrs said:
(which a lot of our servers, with a default DA install, are using),
Click to expand...
Which distro does that? Because on a default DA install I've never seen /home/mysql being used, it was always /var/lib/mysql.
So do Debian/Ubuntu default DA installs use /home/mysql directory??
 
Richard G said:
Which distro does that? Because on a default DA install I've never seen /home/mysql being used, it was always /var/lib/mysql.
So do Debian/Ubuntu default DA installs use /home/mysql directory??
Click to expand...
Not anymore, I think DA changed their setup.sh script for Debian/Ubuntu some time ago, but they used to in the past. So we still have a lot of servers which were installed a few years ago and have been OS upgraded (i.e. Debian 8 -> 11) since, which still use /home/mysql.
 
@smtalk I somehow seems to have created a little mess. Turns out my regular backups now run into this error:

Code: 
RoundCube Backup Error:
Fatal error: Uncaught mysqli_sql_exception: Table 'da_roundcube.users' doesn't exist in engine in /usr/local/directadmin/scripts/backup_roundcube.php:150
Stack trace:
#0 /usr/local/directadmin/scripts/backup_roundcube.php(150): mysqli->query('SELECT * FROM  ...')
#1 {main}
  thrown in /usr/local/directadmin/scripts/backup_roundcube.php on line 150

I don't even use Roundcube by the way, but fair enough. I checked /home/mysql and indeed saw "da_roundcube" in there which wasn't in /var/lib/mysql.

So decided to try to set back the original data as you said:

Code: 
service mysqld stop
mv /var/lib/mysql /var/lib/mysql_backup
ln -s /home/mysql /var/lib/mysql
service mysqld start

However, now mysqld won't start anymore:

Code: 
aug 03 14:16:45 systemd[1]: Starting MySQL database server...
aug 03 14:16:45 systemd[21906]: mysqld.service: Failed to set up mount namespacing: No such file or directory
aug 03 14:16:45 systemd[21906]: mysqld.service: Failed at step NAMESPACE spawning /usr/local/mysql/bin/mysqld: No such file or directory
aug 03 14:16:45 systemd[1]: mysqld.service: Main process exited, code=exited, status=226/NAMESPACE
aug 03 14:16:45 systemd[1]: mysqld.service: Failed with result 'exit-code'.
aug 03 14:16:45 systemd[1]: Failed to start MySQL database server.

I then restored the previous situation and simply copied 'da_roundcube' from /home/mysql to /var/lib/mysql but the backup task will still fail with the same error.

What am I missing here?
 
1. maybe wrong symlink owner
2. it's not enough to copy roundcube database dir, this database must be listed in mysql db_list with described user/pass etc.. so if you don't want to install roundcube - you can try to create just database da_roundcube with table da_roundcube.users. but easier for me - install roundcube and remove it.
 
You must log in or register to reply here.
Back
Top