Incremental Backup using Rsync

M

marvin

Verified User
Joined
Dec 17, 2004
Messages
28
Hi,
I've written an incremental backup script based on Mike Rubel's using rsync. Just hope it'd be useful for others.

Link for the script :
Save it as backup.sh

Directories and Files to be backed up :
Save it as backup_data and put it into /root/scripts/

As you can see from backup_data, the script is not very neat. It works for me however. Any hints for improvements are welcome.


Note :
What are backed up?
/etc/named.conf
/home
/usr/local/directadmin/data
/var/named
 
Sorry... the link down.

</etc/cron.daily/backup.sh>
##### Start #####
#!/bin/bash
#
# 7 daily rotating incremental backup using rsync
# written by marvin
# reference : http://www.mikerubel.org/computers/rsync_snapshots/

unset PATH # suggestion from H. Milz: avoid accidental use of $PATH

# ------------- system commands used by this script --------------------
RM=/bin/rm;
MV=/bin/mv;
CP=/bin/cp;

TOUCH=/bin/touch;
MKDIR=/bin/mkdir;

RSYNC=/usr/bin/rsync;


# ------------- file locations -----------------------------------------

DEST_PATH=/backup;
BCK_PATH=/;
INCLUDES=/etc/backup.data;

# ------------- the script itself --------------------------------------
# rotating snapshots of $BCK_PATH
# create destination directory if not exist
if [ ! -d $DEST_PATH$BCK_PATH ]
then
$MKDIR -p $DEST_PATH$BCK_PATH
fi


# step 1: delete the oldest snapshot, if it exists:
if [ -d $DEST_PATH$BCK_PATH/daily.6 ]
then
$RM -rf $DEST_PATH$BCK_PATH/daily.6
fi


# step 2: shift the middle snapshots(s) back by one, if they exist
let count=5
while [ $count != 0 ]
do
if [ -d $DEST_PATH$BCK_PATH/daily.$count ]
then
$MV $DEST_PATH$BCK_PATH/daily.$count $DEST_PATH$BCK_PATH/daily.$((count+1))
fi
let count=count-1
done


# step 3: make a hard-link-only (except for dirs) copy of the latest snapshot,
# if that exists
if [ -d $DEST_PATH$BCK_PATH/daily.0 ]
then
$CP -al $DEST_PATH$BCK_PATH/daily.0 $DEST_PATH$BCK_PATH/daily.1
fi

# step 4: rsync from the system into the latest snapshot (notice that
# rsync behaves like cp --remove-destination by default, so the destination
# is unlinked first. If it were not so, this would copy over the other
# snapshot(s) too!
$RSYNC \
-va --delete --delete-excluded \
--include-from=$INCLUDES --exclude=** \
$BCK_PATH/ $DEST_PATH$BCK_PATH/daily.0 ;

# step 5: update the mtime of daily.0 to reflect the snapshot time
$TOUCH $DEST_PATH$BCK_PATH/daily.0 ;
##### End #####

</etc/backup.data>
##### Start #####
etc/
etc/exim.conf
etc/named.conf
etc/group
etc/passwd
etc/proftpd.conf
etc/proftpd.passwd
etc/proftpd.vhosts.conf
etc/shadow
etc/system_filter.exim
etc/httpd/
etc/httpd/conf/
etc/httpd/conf/httpd.conf
etc/httpd/conf/ips.conf
etc/mail/
etc/mail/**
etc/virtual/
etc/virtual/**
home/
home/**
usr/
usr/local/
usr/local/directadmin/
usr/local/directadmin/data/
usr/local/directadmin/data/**
var/
var/lib/
var/lib/mysql/
var/lib/mysql/**
var/log/
var/log/**
var/mail/
var/mail/**
var/named/
var/named/**
var/spool/cron/
var/spool/cron/**
var/spool/mail/
var/spool/mail/**
var/spool/virtual/
var/spool/virtual/**
var/www/
var/www/**
##### End #####

Really appreciate if someone could give better idea about backup.data :(
 
Last edited:
Above script is lame. Here's better one.

##### Start #####
#!/bin/bash
#
# x number of daily rotating incremental backup using rsync
# written by marvin
# reference : http://www.mikerubel.org/computers/rsync_snapshots/

unset PATH # suggestion from H. Milz: avoid accidental use of $PATH

# ------------- system commands used by this script --------------------
RM=/bin/rm;
MV=/bin/mv;
CP=/bin/cp;

TOUCH=/bin/touch;
MKDIR=/bin/mkdir;

RSYNC=/usr/bin/rsync;

# ------------- file locations -----------------------------------------
DEST_PATH=/tmp/rsync;
SRC_DIRS="/etc /home /usr/local/frontpage /usr/local/directadmin /var/lib/mysql /var/log /var/named /var/spool/cron
/var/spool/mail /var/spool/virtual /var/www";

# ------------- days of backup -----------------------------------------
DAYS_OF_BACKUP=7;

# ------------- the script itself --------------------------------------
# rotating snapshots of $SRC_DIRS
# create destination directory if not exist

let daysofbackup=$DAYS_OF_BACKUP;


# step 1: delete the oldest snapshot, if it exists:
if [ -d $DEST_PATH/daily.$((daysofbackup-1)) ]
then
$RM -rf $DEST_PATH/daily.$((daysofbackup-1));
fi


# step 2: shift the middle snapshots(s) back by one, if they exist
let count=$((daysofbackup-2))
while [ $count != 0 ]
do
if [ -d $DEST_PATH/daily.$count ]
then
$MV $DEST_PATH/daily.$count $DEST_PATH/daily.$((count+1));
fi
let count=count-1;
done


# step 3: make a hard-link-only (except for dirs) copy of the latest snapshot,
# if that exists
if [ -d $DEST_PATH/daily.0 ]
then
$CP -al $DEST_PATH/daily.0 $DEST_PATH/daily.1;
fi


# step 4: rsync from the system into the latest snapshot
for dir in ${SRC_DIRS}; do
if [ ! -d $DEST_PATH/daily.0$dir ]
then
$MKDIR -p $DEST_PATH/daily.0$dir;
fi
$RSYNC -a --delete $dir/ $DEST_PATH/daily.0$dir;
done


# step 5: update the mtime of daily.0 to reflect the snapshot time
$TOUCH $DEST_PATH/daily.0;

##### End #####
 
Last edited:
Has anyone taken the time yet to see if this backs up everything required?

Jeff
 
I recall that i used DA's directory structure to build the backup directories. However, i have difficulty in finding the reference at help.directadmin.com now.
 
If I backup using this script, in an event of a total hdd crash and after I install the basic OS again, I can restore this backup and everything will be back to normal?

Do I have to install the apps again, like apache, mysql, exim etc...
 
This backup doesn't backup DirectAdmin itself. There may be other things it's missing as well, since it was written some time ago.

I highly recommend you test this, and any other backup system you may be using, before relying on it in a real world restore situation.

Jeff
 
You must log in or register to reply here.
Back
Top