DA backup is so weird ;)

Hi guys,

DA backup is so weird

Because Admin backup in DA is not icremental (it's stupid everyday save and send that same stuff) and causes high load we decide to do it in some clever way.

1. DA system backup (without mysql and users home dir) - for system config files
2. DA Admin backup (without files and mysql) - for user settings
3. mysqlautobackup - for dumping mysql databases
3. rdiff-backup to send 1 & 2 & 3 to another server.

What do you think: will be enough to restore everything in case of emergency? (No?)

Yesterday we want move one of user account to another server and this was excellent time to test this approach. We was shocked when this backup do not restore everything!!!

1. If we uncheck option "Databases" (step 4 Admin backup) then in user.tar.gz file is missing file with info about user DB and pass!!! :O WTF?!?

2. Also we checked "Email accounts" (step 4 Admin backup) but user.tar.gz file also don't have emails!!! (only config)

I'm missing something? How to restore EVERYTHING without make FULL DA backup (like as say before, it's stupid to .gz and send every day that same stuff)?

How do you backup hosting server with hundreds of users?

Thank you for help.

Hello,

With regards to your points:
1) It you uncheck "Databases" from the backups, then it's logical that the databases are not included. This includes the user/pass values.

2) If you uncheck "E-Mail Accounts", this will prevent the email accout names/passwords, as well as their User data from being restore.
The "E-Mail Settings" only refers to filters and catch-all values from the /etc/virtual/domain.com/filter.conf. (hover your mouse over the word next to the checkbox for more info)

-------

Regarding breaking up your backups into smaller parts, you can create multiple cronjobs instead.

Run a daily (or more frequent) backup on things that change very often, but uncheck the items that are large or that don't need backing up as often.
Save it to a path like /home/admin/admin_backups/daily

Then create another backup to be run less often, and uncheck everything, and only include the larger items that you don't need backed up as often (the ones that were not included in the first backup).


When restoring all of this data, you'd restore both files, one after the other.
The order doesn't matter too much, but I'd recommend first restoring the smaller of the 2 bacukps files, and work on the large one with fewer items after.


Sometimes, using something like rsync is more effecient for backups.
Similar to the info above, you'd uncheck the items that are large or very static.
Then use rsync to transfer those items to a remote box every day, as rsync only transfers the data which has changed or is new, which will save you a lot of bandwidth.

Related guides:
http://help.directadmin.com/item.php?id=335
http://help.directadmin.com/item.php?id=298

John

Ive a question about the rsync part John.

Long time ago there was a post that was showing a missing "directive" in the gzip creation that was useful for rsync gzipped file, has been added?

If yes, so after da create the backup can be correctly rsynced elsewhere directly with the gzip format?

Thanks

Thanks for reply!

1) It you uncheck "Databases" from the backups, then it's logical that the databases are not included. This includes the user/pass values.

Click to expand...

No, it's not logical It's logical that .sql is not included but in that case FROM WHERE I can backup database settings (user db/pass)?!?

So if i don't check this then i don't have full backup. It's stupid. This forces me to check this and make sql dumps even if I don't want this

Is any chance to fix this in next release?

Well maybe can be divided in two checks:

MySQL Configurations
MySQL Dumps

What do you think John? Should be easy to implement?

Regards

Hello,

1) The change itself isn't hard for new systems with new skins (just some new variables here and there), however with regards to backwards compatibility issues, it get's very messy, very quickly because all currently active crons would not have the "db configs" option enabled..
Also, any old skins which are not updated would not have the "db configs" checkbox present, thus when they create the backup, no db configs will ever be passed to DA, thus not backed up, unless they have a new skin (which cannot be guaranteed).

It's not impossible, but a decent amount of work with regards to the update checks to go through all old settings to enable the "db configs" if "databases" is enabled in the cron backups.
Then for old skins that don't get updated.. what we could do is have all new skins have a hidden variable like "backup_data_version=1.1".. so if that hidden var is not present, then if "databases" is checked, both "databases" and "db configs" are enabed internally.
If that var is present, then DA assume that form does have "db configs" present, so if it's not passed, it's not missing from the form, but simply unchecked, thus the db configs (user/pass values) are simply left out. (of course, it would usually be be the inverse of that where the database is not included and user/pass is included, but has to work either way)

OR

2) simply add another checkbox called "only db config".. whereby, when it's enabled (along with "databases"), it would leave out the database, but include the user/pass, if both "databases" and "only db config" are enabled.
That would require no backwards compatibility issues, but would also confuse people slightly, and wouldn't be as cleans as a simple option=item setup.

I'd personally prefer to see option 1 as it's cleaner, once implemented... but option 2 would be far easier to add in, since it doesn't have any backwards compatibility issues so saves needing to add update checks and hidden form version numbers.

John

Yes absolutly agree with option 1.

Regarding the compatibility with already present cron.

Im checking a cron right now generated by da admin backup


First backup all, so, if that is selected i suppose mysql dump and mysql conf is selected aswell.
Second is related to everything except database, we should choose if add to this kind of backup databases_confs too or not and use a "sed" on update to change all existing backups.
Third is something random + database, here we can leave as is, setting a rule that say, if option*=database so is saved db and confs, if option*=database_confs we save just confs, if option*=database_dump we save just dumps

So, when both are selected the panel just put "option*=database", if just dump is selected the panel put "option*=database_dump" and if just dbconfs is selected the panel put "option*=database_conf"

Actually, i dont see any reason to save just dump and not confs, so, that maybe should not be used, i mean, if someone select dumps automatically have also conf, but, if e just select confs, dumps are not included.

Regarding the old skins, well, is every single admin own responsability to keep skins up-2-date (if not are the original ones) or contact who sold the skin to get the updates, for example, i had to use ench skin to test login_keys cause the capri skin has not yet been updated and have not that page in it yet, and, ofc, i cant blame da staff for thiss missing page, is not da staff responsability.

I hope i've been enough clear.

Regards

DirectAdmin Support said:

I'd personally prefer to see option 1 as it's cleaner, once implemented...

Click to expand...

I prefer option 1 as well, but when I look above at your notes I can only hope you're not creating code to complex to maintain over the next years.

My suggestion would be to do option 1, but be careful to document it well .

Jeff