rsync with DA backups in /admin_backups

pppplus

pppplus

Verified User
Joined
Dec 19, 2008
Messages
523
Hi

Actually, I copy all backup in /admin_backups in remote servers.
But it takes a lot of time.

Is it possible to use rsync with these backups ? it seems not easy / not possible to use rsync with compressed files.

How do you save datas in remote servers ?
I don't want to same all users directory + mysql with rsync, because it take to much size. copy is ok, but too long.

Thanks for your help and ideas.
 
My recollection is there does seem to be a way to disable compression, but I'm not sure; try searching. Personally, we do daily backups (of most servers) and save a week's backups so users can ask us for a backup from several days ago.

You may be able to save transfer time if you've got an internal network running on it's own ethernet port; that saves time for us.

We also plan on experimenting with saving /home/admin/admin_backups/ as an NFS mount to see if that will save overall time (though it will definitely make initial write much slower there won't be an ft;p transfer time, but I'm not sure how that will work when rotating backup directories.

Jeff
 
Thanks nobaloney.
I already search, but do not find something helping me.

I can maybe disable compression, but it will be a problem of space ! I save 7 days of directadmin backups in a remote server. If it's not compress, it take a lot of space.

I just hope someone already find a solution here, because I'd want to use directadmin backups (it's already backup, so I've only to send on another server).
I do it actually with lftp. It works very good, but is very long ! So I'd want to find a solution for the same thing, but with rsync.
 
Actually yes, it is... the thread started with your same question.. use DA to make local backup and use rsync to remote transfer, and as far as i did read there is the solution for rsync those backups (also compressed).

Did you read the entire thread?

Regards
 
Hi
Yes I see it's start with the same question, but I understand it does something a little different.
Maybe my english is not good enough.

I will read the topic again, thanks.
 
Hello,

1) DA can create the backups tar .tar, instead of .tar.gz (if you want)
http://www.directadmin.com/features.php?id=1220

2) SeLLeRoNe's reference to thread 33357 would make the tar.gz files "rsyncable", so that rsync only sends over a small portion

3) If space is an issue and you can't even create the bacukps in the first place, then use:
Admin Level -> Admin Backup/Transfer
but in "Step 4: What", you can de-select "email data" (leave email accounts), and de-select "Domains directory".
Then you can use rsync to just transfer over the entire /home folder to anthother server.
It would use very little bandwidth, doesn't create another copy locally, and only transfers what changed.

John
 
Ok sellerone, I read again the entire post, and try it.
It's ok, it does what I want, thanks for your help.

Just one "problem" .tar are bigger than .tar.gz (1.5 times to 10 times !)
I test only in small files, and I hope big files does not have problems...

But it does what I want. Now, it would be better if rsync work with .tar.gz

John, maybe I will modify da conf to save directly in .tar.
But I have to verify, if user backups are in .tar or .tar.gz (I prefer they have .tar.gz for space on their account).
But I don't want to use your method, because it's one more backup script, and I store 10 days of backups, so it will be too large.
 
There's a way to do this without any adjustments on your server/configs...
Get more backupspace or drop then on a compression/deduplication enabled ZFS backupstorage.
:D
 
sysdev, can you give me more informations about your proposition ?
 
or simply add an NFS mount i.e /my_backups, and have DA to backups to that directory.
 
pppplus said:
Ok sellerone, I read again the entire post, and try it.
It's ok, it does what I want, thanks for your help.

Just one "problem" .tar are bigger than .tar.gz (1.5 times to 10 times !)
I test only in small files, and I hope big files does not have problems...

But it does what I want. Now, it would be better if rsync work with .tar.gz

John, maybe I will modify da conf to save directly in .tar.
But I have to verify, if user backups are in .tar or .tar.gz (I prefer they have .tar.gz for space on their account).
But I don't want to use your method, because it's one more backup script, and I store 10 days of backups, so it will be too large.
Click to expand...

I'm not sure if I follow, but a while back I started a thread with rsync+.tar.gz. If you apply the modification to the cron file as discussed in the thread, you can rsync .tar.gz backups made by DA.
 
A while ago I tried to make this process less bandwidth consuming. You can create a daily DA backup of all databases (they are most likely the smallest backup files), and then you can just backup the /home/ folder with rsnapshot (incremental rsync) from an external server or NAS.

One of the downsides of this method is that you have to connect from your NAS or your backup server through a public RSA key without a password on a user with root access. That could be a security issue, but it's not that this method is completely unsafe to use.

Files will get copied through a secure connection in stead of the much more insecure FTP (default option in DA backup/restore) where password and accountname are send in plain text.
 
ericovk said:
Files will get copied through a secure connection in stead of the much more insecure FTP (default option in DA backup/restore) where password and accountname are send in plain text.
Click to expand...
As long as you recognize that a secure connection means data encryption which uses processor time on both ends of the connection, and therefore makes the process run more slowly. Security is always atradeoff. I might consider the tradeoff acceptable for connections over the 'net but not as important within (for example) one cabinet, through a router, using subnets, in which case I might depend on routing and subnet security.

Jeff
 
It's always good to test. We have a heavily loaded server in which we're trying to speed up backups. We thought that we could speed up backups over our 100-mbps lan by turning off compression of the backp files on the server and compressing them later on the backup server, which has machine cyles to spare. I thought the speed over the 100-mbps lan was fast enough so the larger file size wouldn't be an issue.

But from start to finish, the uncompressed backup took 2.5 times as long as a compressed backup.

I decided to solve the problem with hardware; we're building out a new server with 3x the resources and a separate drive for /home/tmp.

Jeff
 
You must log in or register to reply here.
Back
Top