full backup

tritting

Verified User
Joined
Oct 10, 2004
Messages
12
As an interim solution, you may want to consider Reoback to pickup, backup, and transfer any files/dirs you feel are being missed by DA. We used this simple script via cron for quite some time with good results.

Be well, TR
 

nobaloney

NoBaloney Internet Svcs - In Memoriam †
Joined
Jun 16, 2003
Messages
26,119
Location
California
Chrysalis said:
give me a pm of what you need.
I'm still far behind in a lot of work, so it would be a while :( .

But I will probably be publishing the list of files, directories, and paths required for a successful reinstall on RHL by the end of the week, and then perhaps one of the FreeBSD gurus can figure it out.

Jeff
 

nobaloney

NoBaloney Internet Svcs - In Memoriam †
Joined
Jun 16, 2003
Messages
26,119
Location
California
tritting said:
As an interim solution, you may want to consider Reoback to pickup, backup, and transfer any files/dirs you feel are being missed by DA.
There's no problem with the sysbk script supplied by DA; the problem is with the list of directories and files it's supposed to back up.
We used this simple script via cron for quite some time with good results.
You don't know how successful you are until you've tried a complete restore to a new box without referring back to the old one (in other words a worst-case scenario restore).

It's with the restore that we found problems with the file/directory list.

If you know you have a file/directory list that will create a worst-case scenario restore, then why don't you share?

Thanks.

Jeff
 

Chrysalis

Verified User
Joined
Aug 25, 2004
Messages
1,584
Location
uk
yeah I would agree the directadmin backup tool is quite excellent its just the list of what it backs up is incomplete, and the restore is lacking.
 

tritting

Verified User
Joined
Oct 10, 2004
Messages
12
jlasman said:
There's no problem with the sysbk script supplied by DA; the problem is with the list of directories and files it's supposed to back up.
I didn't think I implied any different; as my post suggested, Reoback is simply a possible intrim solution to grab any files/dirs that are being missed by the DA backup routine until the issue is addressed.


You don't know how successful you are until you've tried a complete restore to a new box without referring back to the old one (in other words a worst-case scenario restore).

It's with the restore that we found problems with the file/directory list.
No argument on this point. Here's how we minimized the impact:
1) I don't believe in automated restores.
2) All boxes when completely setup (including all user accounts) were imaged (dd works quite well for this task) and the image stored.
3) Any custom additions added after the imaging were added to the Reoback file list.
4) /var /etc and certain /usr locations backed up nightly (FTP over SSH to remote location; this way I CAN refer to the old box).
5) Major software upgrades or security patches documented and filed for future reference.
6) Customers responsible for /home dirs. (however, we did back them up from time to time)
7) Dreamed nightly of being able to afford a nice Raid array. ;)

We did get lucky and only had 2 failures over 6 servers in 9 years, but restores were a piece of cake.

Be well, TR
 

nobaloney

NoBaloney Internet Svcs - In Memoriam †
Joined
Jun 16, 2003
Messages
26,119
Location
California
I don't see why you'd need Reoback to do that; you can just add the same directories and files to the sysbk backup list.

Jeff
 

tritting

Verified User
Joined
Oct 10, 2004
Messages
12
My impression of this thread was that there were files and dirs being left out of the list, and to have them included was something in the DA backend and hence the reason for my Reoback suggestion until this was accomplished by the DA folks. Apologies to all if I've misunderstood...

Be well, TR
 

nobaloney

NoBaloney Internet Svcs - In Memoriam †
Joined
Jun 16, 2003
Messages
26,119
Location
California
Point taken.

The list of files and directories is customizable by the admin user in the DA interface.

Jeff
 

Chrysalis

Verified User
Joined
Aug 25, 2004
Messages
1,584
Location
uk
I got a new problem with the backup now, it is crontabbed for every morning at 4am but it isn't finishing it stops at the mysql stage, the logging just stops as well. look below

---

Performing sanity checks: Completed
Checking load average: Completed
Checking free disk space: Completed
Performing Custom backup
Archiving /etc/exim.conf: Completed
Archiving /etc/exim.crt: Completed
Archiving /etc/exim.key: Completed
Archiving /etc/exim.pl: Completed
Archiving /etc/group: Completed
Archiving /etc/hosts: Completed
Archiving /etc/passwd: Completed
Archiving /etc/proftpd.conf: Completed
Archiving /etc/proftpd.passwd: Completed
Archiving /etc/proftpd.vhosts.conf: Completed
Archiving /etc/resolv.conf: Completed
Archiving /etc/shadow: Completed
Archiving /etc/system_filter.exim: Completed
Archiving /etc: Completed
Archiving /etc/mail: Completed
Archiving /etc/virtual: Completed
Archiving /usr/home: Completed
Archiving /usr/local/directadmin: Completed
Archiving /usr/local/etc: Completed
Archiving /usr/local/frontpage: Completed
Archiving /var/log: Completed
Archiving /var/mail: Completed
Archiving /var/spool/cron: Completed
Archiving /var/spool/mail: Completed
Archiving /var/spool/virtual: Completed
Archiving /var/www: Completed
Performing DNS backup
Stage 1 DNS backup: Completed
Stage 2 DNS backup: Completed
Performing Apache backup: Completed
Performing MySQL backup
Stage 1 MySQL backup:<br>

------

thats where it ended.

yet if I do a backup by hitting backup now it works every time.
 

Chrysalis

Verified User
Joined
Aug 25, 2004
Messages
1,584
Location
uk
sorry heres the bit that cutoff

Performing MySQL backup
Stage 1 MySQL backup:MySQL could not be shutdown, aborting...
Performing cleanup operations: Completed
 

nobaloney

NoBaloney Internet Svcs - In Memoriam †
Joined
Jun 16, 2003
Messages
26,119
Location
California
If your server doing it's crontab.daily stuff at the same time?

If so then perhaps for some reason mysql is already shut down and the error is causing it to abort.

Note that this is a guess.

Jeff
 

Chrysalis

Verified User
Joined
Aug 25, 2004
Messages
1,584
Location
uk
it worked this morning.

I got 1 other crontab for 4am the license check.

0 4 * * * root echo 'action=check&value=license' >> /usr/local/directadmin/data/task.queue
 

Chrysalis

Verified User
Joined
Aug 25, 2004
Messages
1,584
Location
uk
well I moved the crontab to 4.30 am and noticed it wasnt working but also it wasn't seemingly even trying to work.

So I did a manual backup and seen gzip start in the process list, but now 5 mins later nothing is happening.

1 - There is no new dir for today's date in the backup dir
2 - There is no message for me in directadmin
3 - 'View last backup log' Shows the log from my last succesful backup 5 days ago.
4 - There is plenty of free disk space (well over 90gig)

any idea what could be wrong?
 

Chrysalis

Verified User
Joined
Aug 25, 2004
Messages
1,584
Location
uk
sorry it was working, I told it to delete local file after remote transfer and the dirs exist on the remote server.

But I am getting no backup done messages generated when it finishes.
 

ClayRabbit

Verified User
Joined
Jan 3, 2004
Messages
260
Location
Russia
Damn... That DA feature killed my old sysbk config...
It seems their new features becomes more and more painfull for me...

We'll never use that feature because we already using modified version of sysbk to backup config files and mysql databases.
Furthermore, we'll never use sysbk for backing up user's home directories. It's great amount of data to copy over and over every night - so we implemented script for some kind of incremental backup to backup only new/modified data from /home/*.

A list of directories we are backing up with sysbk:
/etc/ssh
/etc/virtual
/usr/local/directadmin/conf
/usr/local/directadmin/data
/usr/local/directadmin/scripts/custom
/var/spool/cron (/var/cron on FreeBSD)
/var/spool/mail (/var/mail on FreeBSD)
/var/spool/virtual

List of files:
/etc/passwd*
/etc/group*
/etc/shadow*
/etc/*.conf
/etc/*.cnf
/etc/proftpd.*
/usr/local/frontpage/*.cnf

(There is no /etc/httpd/conf and /var/named in this list while that files are handled individually by sysbk.)

I believe this set of files is enough to completely restore web-server on new machine. But maybe I have missed something :)

Anyway backup lists need to be customized for your own systems - all depends on which files you are modifying, so maybe you have important and exclusive data in other locations of your hdd ;)

Maybe you'll prefer also to include directories:
/etc/cron.d
/etc/cron.daily
/etc/cron.hourly
/etc/cron.weekly
/etc/cron.monthly
/etc/mail
/root/.ssh
/usr/local/directadmin/plugins
/var/www

and files:
/etc/crontab
/etc/exim.*
/etc/hosts
/root/.*

PS: I wonder why I still use sysbk while all I need is creating tarballs from list of files and directories?... We are not using remote backups so I think there is also no point to use md5sum. As for mysql backup - I already rewrited this sysbk module to make it with mysqlhotcopy.
BTW, why sysbk stores 2 copies of mysql in different format? And both ways are not best - direct copy need to shutdown server, and mysqldump is quite slow epecially on restore.
 
Last edited:
Top