No backups durring the DAY !!!

S

Salmus

Verified User
Joined
Feb 15, 2008
Messages
39
Hello,

I have 15 clients with data on account ~ 15 GB - and they have a stupid need to make backups in the middle of the day - or every evening ! (18:00 -> 22:00 P.M) When my servers are really requested - and they cannot handle 2 concurrent backups with more than 1500 requests per sec.


There is any possibility to set DirectAdmin to queue backups for the midnight for example ? Or - in the interval 02:00 -> 05:00 A.M when all the servers are more idle than busy ?

Please ! I really need this ...
 
In other words, force them to not run backups when they want to, but when you want them to?

Jeff
 
In other words: force backups (queue) to be made at specific range of time

For example: between 02:00 -> 05:00 A.M. - when a major of servers are not used so much.


An easy fix for me will be to cat /usr/local/directadmin/data/task.queue end extract lines with "backup" word - and reinsert them in the specific range of time (via cron). But an configuration option in DirectAdmin about this will be awesome.


I do not think that I'm the only one who needs this, no one will want that backups to be made in the middle of day day ... when load is high.
 
Last edited:
Great idea, so far I've never run into this problem, but I've seen clients do backups during "work" hours (when they are at the office) and the server is busier, but a queue for midnight would rock.
 
Hmm.. for the backup cron creation, the all_pre.sh would be able to do this... but for the "now" option (immediate backup), it would be more complex. It would be a bit confusing if they issued a "now" backup.. DA accpeted it, but then did nothing for several hours.. I could see that being a bit confusing. Another option would be to have backups placed into a backup.queue file.. and have a dataskq cronjob executed whenever you want to process that queue.. there are several ways of going about it that could be considered.

John
 
Well usually its not a problem if a user wants to do a now backup. But I do understand if you have a customer with like alot of files. It could put alot of load on the server at the wrong time.

Another idea might be to renice the backup with extremly low priority.
 
I believe it's already running with an extremely low priority.

Jeff
 
I think it's easier to run a cronjob and renice the process if load is e.g. >4.
 
Unfortunately this won't solve all the problems.
For example I've a couple MyISAM DB with 1-2GB of data; mysqldump seems to have exclusive access when dumping MyISAM data, therefore any page or cron script can't access the DB while the backup is running.
 
Renice-ing the process will not solve the problem ... MySQL is the biggest problem - it will bring the load upto 15 on my machines for databases ~ 1,5 GB - and it will drastically increase response time of MySQL.
 
Here's just a quick sample of blocking end Users (Site Backup) from running backups during the day. This example requires the time to be after 10pm and before 6am.

Create:
/usr/local/directadmin/scripts/custom/all_pre.sh

Add the code:
Code: 
#!/bin/sh
if [ "$command" = "/CMD_SITE_BACKUP" ]; then
        NOW=`date '+%H'`
        EVE=22
        MORN=6

        if [ "$EVE" -le "$NOW" ] || [ "$MORN" -gt "$NOW" ]; then
                exit 0;
        else
                echo "please run your backups between hour $EVE pm and $MORN am";
                exit 1;
        fi
fi
exit 0;
chmod the all_pre.sh to 755, and you're in business.

This doesn't address Reseller Backups, specifically cronjobs.. but the same idea could be used for cron creation.. check the variables for the hour set for the cron to ensure it's within the range you want.

Edit: thinking about it, you can use this same concept... if the time is during the day, then still return "exit 1", but with a "backup queued" message, and you could add the username to a list, and have another cronjob to execute the list later that night.. using the task.queue format you see when you create a backup normally... just dump the same command to the task.queue when you want, and you've now got queued backups during the day, and instant ones during the night.

John
 
It's a nice idea, but why can't you just set the priority for the backups to be as low as possible? That way it may take a few extra minutes (or hours even) to complete, but it would not be impacting your server's ability to handle requests.
 
It already is set to 19 I think... the problem is that any connection to mysql doesn't have a nice value.. mysqld controlls the priority, not mysqldump (unless there is a way to set mysqld via mysqldump through the socket).. because the load isn't myqldump that's spiking the cpu usage.. it's the mysqld process which is a seperate entity that is already running.. The dump is treated like a connection just like when viewing a forum... except all data is requested at the same time.
Related: http://www.directadmin.com/features.php?id=577

John
 
It's already set to 19.. that's the default.
If you want to change it to something other than 19, then you add it to the directadmin.conf (it overrides the internal default).
To see what you've got right now, type:
Code: 
cd /usr/local/directadmin/
./directadmin c | grep backup_nice
and that shows you what's being used (assuming DA was restarted if you've made a change)

John
 
Thanks John.
It would be nice to have some kind of doc that tell us what we can do, how it works and why. Maybe there is one but in that case I havn't been able to find it.
 
To see all options, type:
Code: 
cd /usr/local/directadmin
./directadmin c
Then simply type in the variable name into:
http://www.directadmin.com/versions.php
And you'll get the docs.

The versions system holds all feature/variables/changes ever done... while the help.directadmin.com contains guides on how to use some of the more commonly used features (faq).

John
 
You must log in or register to reply here.
Back
Top