Advice for backup solutions?

C

Cutia

Verified User
Joined
Jul 31, 2004
Messages
41
Hi, I'm interested in finding the best solution to do complete system backups for my remote server. I'm thinking of installing a second hard drive in the box and doing nightly backups to it, using it as a mirror of the main HD. Of course I have no idea how I can then use this second HD if the first one fails. I presume I contact LayeredTech who I have the server with and ask them to boot off the backup HD.

I'm also keen to have some kind of rollback system if possible to undo system upgrades that might have caused problems. The main difficulty with all of this is the remote aspect I think.

Does anyone have any experience with this kind of thing?
 
We've learned what to do, what not to do, and what's definitely not okay to do :) .

If you're going to use the two-drive system you have to have two completely bootable drives, and some method of copying them to each other. Not hard, but not simple. There are lots of tutorials out there for doing that.

Then you don't even have to swap the drives except when you're ready to actually replace the bad one ... just disable the bad one in the bios and let the bios find the second one. Note you may need to make custom changes to lilo or grub.

Jeff
 
Thanks Jeff, your advice is appreciated. My main concern is how to do anything like that remotely. I'm guessing I'd have to put a request in with my provider LayeredTech to go into the bios and disable the dead drive.

LT recommend using a RAID1 array of two drives for a no hassle backup solution. This is ok but it means renting a RAID card which is not that cheap.

What do you do when you upgrade software? Do you have some sort of backup so you can rollback in case of problems?
 
Cutia said:
Thanks Jeff, your advice is appreciated. My main concern is how to do anything like that remotely. I'm guessing I'd have to put a request in with my provider LayeredTech to go into the bios and disable the dead drive.
Click to expand...
You could do that. You could also have them install a remote KVM solution for you, or if your server has it available in the BIOS, set up the server so all startup commands (including the BIOS change command) go out the serial port, and then install a serial connection you can use from your desktop.
LT recommend using a RAID1 array of two drives for a no hassle backup solution.
Click to expand...
If they really said no hassle then they're being over simplistic and perhaps their recommendations shouldn't be trusted without discussing everthing with them in detail first :) .

For example, I'd ask them how often RAID fails for spurious reasons compared to actual drive failure (answer: much more often).

I'd ask them how often RAID fails and both drives are bad, compared to how often only one drive is bad (answer: at least occasionally).

(Sometimes RAID can actually take down the second drive when the first fails. It's known to happen.)

We recommend RAID for everyone, but as part of a solution, not a complete or only solution.
This is ok but it means renting a RAID card which is not that cheap.
Click to expand...
I can't speak for FreeBSD, but on Linux that's arguable; many of us believe that software RAID is a better solution than hardware RAID.

Note however that the default LILO/GRUB installations do NOT work when the first drive in a RAID configuration fails. It took us about a year and at least one failure before we figured out how to fix it so it would work.

What do you do when you upgrade software?
Click to expand...
We update first on a testbed solution and study the upgrade results before we upgrade other systems.

RPM / YUM / Up2Date generally doesn't break anything except occasonally configuration files. So configuration files are the first thing we check if we have problems with a daemon after an update.

CustomApache generally doesn't break anything; don't forget it had to run at least once, in order for DA to run on your server.

Let me rephrase: CustomApache generally doesn't break anything unless you've made custom changes inside it. When we do that, we do it first on a testbed server.

Services that have to updated manually because they're covered neither by RPM/YUM/Up2Date or by CustomApache, we create new versions in their own directories; then we create symbolic links for the various versions. If the link to the new version doesn't work we create a link back to the old version until we figure out why the new version doesn't work.
Do you have some sort of backup so you can rollback in case of problems?
Click to expand...
We backup our retail account servers daily, our dedicated servers weekly or more often as contracted, and our customer services not in our data center depending on what they want and pay for.

Yes, it can easily take multiple testbed servers and terrabytes of backup space. Anyone who told you that webhosting was a cheap business to get into was wrong ;) .

Jeff
 
Wow, thanks for all that info. Yep, looks like an expensive business...
 
As long as your on the "Backup" question, can DA find a second HD if I install one?. My desire is to create a place to backup a different servers accounts, so from two servers, each would keep a backup of the other, And I wanted to try the Admin Backup/Transfer BETA system for testing, seems like a reasonable solution for redundancy. needed would be ftp access to the second drive.....
 
By default DA doesn't know a thing about the second drive. Linux doesn't think of drives as drives, but rather allows you to mount them on a partition.

You can create an empty directory /home/backup, and install a new drive, partitioning it with one partition.

Then mount that partition as /home/backup, and automate the procedure.

All of which is generic systems administration stuff.

Once it's there, you can tell the DA backup system to write it's backups to /home/backup.

Jeff
 
For those who do not have the priviledge of doing complete HD backups, my backup schedule has worked very well for me.

DA does a full /home backup, as well as all of the config files, on a weekly basis. I then download the tar.gz's onto my own computer and also leave them on the server themselves.

I also download a mysql db dump of the billing/helpdesk software nightly automatically via a cronjob on the box, and a "scheduled task" on my own computer.

I have yet to figure out how to make the simple FTP program built into windows download an entire directory and all of its contents, so the weekly backups are downloaded manually.
 
Your backup schedule works, but have you tried a restore to make sure you have all the files you need to do a restore?

Test restores are very important.

Jeff
 
You must log in or register to reply here.
Back
Top