RSync scripts

D

dreamline

Verified User
Joined
Dec 3, 2005
Messages
108
Hi All,
I just finished my RSync backup script and incase somebody might want to know how to go about that, here's my backup script and exclusion list.

First the backup script:
Code: 
#!/bin/sh
#
# (c) 2010 dreamline
#
# Crontab:
# 30 11,22 * * * <path to script>/rsync_backup.sh > /home/<username>/backup_log.txt
# 
#
# Some RSync parameters explained:
# a = archive
# b = make backups
# E = preserve executability
# h = output numbers in a human-readable format
# v = verbose
# z = compress
# l = keep symlink
# L = copy symlink reference
# r = recurse into dirs
# m = remove empty dirs
# n = dry-run
# p = keep permissions
# t = keep modification times
# g = preserve groups
# o = preserve owner
# u = skip files that are newer on the receiver

SRC="/"
DST="/home/<username>"

/usr/bin/rsync -lavzrmpu --force --delete --delete-before --delete-excluded --exclude-from <path to script>/rsync_backuplist.txt "$SRC" "$DST"
chown -R <user>:<user> $DST # Here I chown everything to the chrooted user
chmod -R 744 $DST # Only give full access to chrooted user, others get read-only

Here's my exclusion/inclusion script:
Code: 
##################################################################### 
#
# (c) 2010 dreamline
#
####################################################################
# Backup complete /etc directory
####################################################################
+ /etc**

####################################################################
# Backup MySQL and Admin domains
####################################################################
+ /home
+ /home/mysql**
+ /home/admin**

####################################################################
# Backup DirectAdmin (do not backup mysql.conf because it contains
  your da_admin password, also exclude updates and custombuild)
####################################################################
+ /usr
+ /usr/local
+ /usr/local/directadmin
+ /usr/local/directadmin/conf
- /usr/local/directadmin/conf/mysql*
- /usr/local/directadmin/custombuild*
- /usr/local/directadmin/updates*
+ /usr/local/directadmin/**

####################################################################
# Backup php.ini (PHP)
####################################################################
+ /usr/local/lib
+ /usr/local/lib/php.ini

####################################################################
# Backup config.inc.php (phpMyAdmin)
####################################################################
+ /var
+ /var/www
+ /var/www/html
+ /var/www/html/phpMyAdmin
+ /var/www/html/phpMyAdmin/config.inc.php

####################################################################
# Backup complete /root directory
####################################################################
+ /root**

####################################################################
# Exclude all the rest
####################################################################
- *

Hope this helps someone. :) However if you plan on syncing this backup through rsync with your local PC then I suggest you use a chrooted user which is locked in it's own home directory. That's where I create my rsync backup and from there I retrieve it to my windows box using Rsync too by using a public-private ssh key. :) Still testing the windows part of rsync. :)
 
Last edited:
should i suggest you to include custombuild and template customization?

/usr/local/directadmin/data/template/custom/
and
/usr/local/directadmin/custombuild/custom/

for have all the template customization you made on your server.

Ive a question also, where do you put the exclusion list? I mean, in the first script doesnt call the second one, so, how do you backup all?

Thanks
 
Well the list here is an include and exclude script. Look closely and you see + and - at the start of the lines. The - is exclude from the backup and + is include in the backup. :)

So the complete DA directory is backed up except for custombuild, updates and mysql.conf.

At the end you see - * which basically says exclude everything from the backup. So if I didn't have any other directives (before the - *) then nothing would be backed up. :)

** means include/exclude directories, subdirectories and files
* include /exclude file or directory (do not recurse into other dirs when used on a directory)

Oh the second script should be named rsync_backuplist.txt. Which is mentioned in the first script by --exclude-from <path to>/rsync_backuplist.txt . :)
 
Last edited:
yeah thanks i noticed that time later and i didnt post :)

But, on exlcusion list you exlude custombuild
- /usr/local/directadmin/custombuild*

so, this will exclude subfolder aswell, and, the custom subfolder should be nice to backup i think.

Btw, sorry for those idiot questions, but does script are on the server you wanna backup right?

So my question is, this is a local backup? Cause i dont see any ftp/remote connection for uplaod those file, am i right?

Ive read you use another rsync instance on your pc for "import" the backup, but, why dont you upload directly from the script?

Regards
 
Hi SeLLeRoNe,
I excluded custombuild because I only use that to build Apache, MySQL and PHP and a few others, however I don't know what's in the custombuild custom folder though, never looked at it. :) If you want to include the custombuild customfolder only then it would look something like this:
+ /usr/local/directadmin/custombuild
+ /usr/local/directadmin/custombuild/custom**
That way the customfolder and all it's content would be backed up too.

I run the script as a cronjob on my server. It then copies the files to a chrooted user on the same server. At home I run Windows, so I had to compile Rsync with cygwin. Then I looked at cwRSync for some settings and needed dll's. Next I created a dos script which connects to my server with the chrooted user and a public-private ssh key. Then it syncs a location on my harddrive.

Now I hear you ask why do you use RSync locally and not from the server. That question is easy to answer: My Server has a fixed IP while my PC at home may change IPs from time to time which might mess up the syncronization if I have rsync on my server connect to my local pc. On my local PC I run a scheduled task to start the windows rsync script which then synchronizes with my server.

Ps. No problem asking questions.


Oh I noticed an other thing from you post:
/usr/local/directadmin/data/template/custom/
The full directadmin folder is included in the backup, only the following files/directories are excluded:
- /usr/local/directadmin/conf/mysql*
- /usr/local/directadmin/custombuild*
- /usr/local/directadmin/updates*
All the other files/directories in directadmin are included in the backup. :)

:)
 
Last edited:
Thanks again for your explanation.

I would suggest you to use dyndns.org for dynamic ip at home (many routers now support dyndns)
This would permit you to set a host on your server and connect always to your home cause the router that support dyndns will send the new ip everytime it change ;)

I use to do this way to have always my home ip for connect to it.

BTW. ive already added this line to the exclusion list:
+ /usr/local/directadmin/custombuild/custom**
Does this work or your way should work better?

Regards
 
Im not expert on rsync so i dont really know hwo to set to send via ftp all the data, so, should you please point me somewhere?

Thanks
 
Well, actually I haven't gotten into rsyncs feature to sync remote yet. I'm definately not an expert on rsync yet, but in a couple of days I figured out how to use rsync to meet my needs.

However I might dive into remote backing up since I do have 2 servers. I can always see if I can create a cross backup. :) So a backup from server 1 will be stored on server 2 and vice versa. But at the moment I haven't tried that yet. :(
 
Last edited:
Hello,

Rsync is neither using, nor supporting SSL. Am I right? If so, how do you protect your data (of course if you do any protection) while transferring in Internet?

Do you use any VPN or secured tunnels? I've found this article and want to get your opinion.

Regards,
Alex.
 
You must log in or register to reply here.
Back
Top