Merging multiple servers into one

[Evo--]

Hi,

I've got a few old DA servers that I'd like to migrate to new servers. However with new servers comes more power, so we'd like to move multiple old servers to a new one.
Now the problem is, that making a backup of all users on the old server takes a while, and during the backup/restore procedure, it would be better to keep a site offline to limit inconsistencies. Normally this would be done with rsync, but because we're merging several servers, this will probably fail due to /etc/passwd mismatches.

So a backup of all users doesn't sound like a option, but per user doesn't sound to great either. We could make a backup of an individual user, move it, restore it.

Would this be a sane solution? Is there a way to automatically restore a backup? Or are there already some script that do something like this?

thanks in advance,
Rogier

 

[nobaloney]

You're almost always going to be required to accept some downtime, becuse of problems you'll have with database driven sites if you don't.

I'd recommend backing up and restoring single resellers at a time using the admin reseller backup.

Yes, it takes time. If anyone has a better idea, I'd like to know it.

Jeff

 

[ranz]

I'm also interested in knowing another way ... so will watch this thread.

I have thought about the possibility of putting both DA boxes behind a NAT. Then when all the appropriate accounts have been moved, change the iptables rules to point to the new DA box - which will result in an instant change to where the "traffic" goes to. I'm still trying to get my head around exactly "how" I would do this ... and am thinking that two public IP's will be needed (at least). Obviously you'd make the DNS changes first. As I said, it's just an idea - and needs investigating ...

 

[floyd]

Ranz, that still does not affect the downtime required to do the backups which is the main problem. Domain/ip resolution is not a problem if you change the TTL's a few days prior to the move.

The user needs to be suspended while the backup is taking place. Unless there is GB's of data then the downtime is usually a few minutes.

I thought I had posted a detailed procedure before somewhere here. If nobody can find it maybe I will put it in the how-to section.

 

[ranz]

yeah I remember reading your excellent post - and think that it definitely should be considered for a "sticky howto".

 

[Evo--]

The user needs to be suspended while the backup is taking place. Unless there is GB's of data then the downtime is usually a few minutes.

That's the problem alright. Now I don't mind a few minutes downtime per user, but with 300 users, it's going to take a while.

I thought I had posted a detailed procedure before somewhere here. If nobody can find it maybe I will put it in the how-to section.

Couldn't find it, sorry.
Does it also describe how to automate the process? Or is there a scriptable way to restore backups? (Couldn't find that either)

 

[floyd]

Couldn't find it, sorry.

It only took me a couple of minutes. Search the forum for posts made by user floyd with the keyword backup.

http://www.directadmin.com/forum/showthread.php?t=36215

Now I don't mind a few minutes downtime per user, but with 300 users, it's going to take a while.

You don't suspend all 300. You only transfer a few, or one, at a time. Start with the resellers and then the users.

Its going to take a while no matter how you look at it.

If you write a script to do it:
1. Use the DA API to suspend
2. Write the backup command to task.queue
3. Detect when the backup is finished
4. Transfer the back up to the new server
5. Change the ip on the old server for the user's zone files
6. Another script on the new server will need to detect that the backup file is present and complete.
7. Write the restore command
8. Use the DA API to unsuspend when the restore is finished

 

[Evo--]

You don't suspend all 300. You only transfer a few, or one, at a time. Start with the resellers and then the users.

Its going to take a while no matter how you look at it.

If you write a script to do it:
1. Use the DA API to suspend
2. Write the backup command to task.queue
3. Detect when the backup is finished
4. Transfer the back up to the new server
5. Change the ip on the old server for the user's zone files
6. Another script on the new server will need to detect that the backup file is present and complete.
7. Write the restore command
8. Use the DA API to unsuspend when the restore is finished

Check, that was what I was thinking about. IIRC you can also tell the backupscommand to transfer it.
The script to detect wether the backupfile is completed isn't that hard, made something a few years ago. When I'm done, I'll put it online.

 

[floyd]

You also do not want to start the next backup process until the previous one is complete.

 

[Evo--]

I suppose the queueing system is clever enough?
Just put several suspend and backup commands in the task.queue and DA will process them in that order, right?

Same goes for the receiving side. When my script notices there's a new backup in it's folder, it will put a command in the queue and I'd think DA would first finish the first before processing the second.