Suggestions needed: Multiserver help

congkai

congkai

Verified User
Joined
Dec 12, 2005
Messages
142
Location
Singapore
Hi people,

I am soon setting up a 2nd server (to act as a backup or emergency server). I need some suggestions. The aim is to get LOWEST downtime possible in a event of a harddisk or server crash. Take note: both server is in a different data center.

Take Server A as the main server, Server B as backup server.

i have a few ideas. i dont know which is the best. I need experts like you all to give me ideas.

NS1.myserver.com and NS2.myserver.com (dummy domain), can i put ns1.myserver.com as my main server A and ns1.myserver.com as server B. Does nameserver works like as if NS1 is down (no respond) it will go to NS2 immediately?

I read up the multiserver of DA. and i notice that it only does the DNS synconise and NOT THE USER FILES. so thinking of how to make both server has the most recent updated files, i think of using the Admin Backup/Transfer and send over the backup files by FTP.

I was thinking of using FTP to transfer the files over, and everyday i will go in the server to restore the user files so that all account settings, emails, user files and everything will be up to date (i know that the user can login to both server if user knows i am doing this, but its okay ignore this for now).

I was also thinking of using Rsync (i am totally newbie to Rsync, in fact, i have never used it before). Using Rsync to synchronize user account files and setting and emails and almost everything, (if i use rsync, i wont have to login to restore those .tar files everyday). Am i right to say this?

I was trying to think of the best plan to have lest down time assuming my 1st server is down.

Which is the best way? :confused:


oh for experts out there, if you can be my help over DA at times, please add me in live messenger. address is stated on the msn icon beside my name THANKS!
 
Last edited:
congkai said:
I am soon setting up a 2nd server (to act as a backup or emergency server). I need some suggestions. The aim is to get LOWEST downtime possible in a event of a harddisk or server crash. Take note: both server is in a different data center.
Click to expand...
You want a failover server (this is the right name), but since your servers are in two different datacenters it won't be very efficient: you can't have the same IP address in both machines and this will make things very difficult and slow when deploying the failover server.
If you have traffic limitations it won't even be very cleaver: keeping two servers synchronized takes a lot of bandwith and you would end up using all of (or, worst, more than) your paid traffic.

If those datacenters are connected one another (adiacent AS, one hop away) you may arrange with the respective support teams to route the address of one machine to the other too when requested, and to have unlimited traffic from one server to the other... but they may want to charge you for this.

It would be way easier to have two servers in the same rack, or even one more (very small, cheap but robust) to run keepalived and control the two others as a LVS cluster.

congkai said:
NS1.myserver.com and NS2.myserver.com (dummy domain), can i put ns1.myserver.com as my main server A and ns1.myserver.com as server B. Does nameserver works like as if NS1 is down (no respond) it will go to NS2 immediately?
Click to expand...
Yes, but it won't help much if the services offered by server A and server B are on different IP addresses, because you will have to change DNS records anyway and it will take hours, for some ISPs even more.
I suggest doing that anyway for load balancing, but again it won't help in case of a disk failure on A.

congkai said:
I read up the multiserver of DA. and i notice that it only does the DNS synconise and NOT THE USER FILES.
Click to expand...
Yep, and it's because of DNS RFC, that asks for nameservers to be on different subnets and servers because of load balancing and to avoid bad caching. Nothing to do with web or email services failover.

congkai said:
so thinking of how to make both server has the most recent updated files, i think of using the Admin Backup/Transfer and send over the backup files by FTP.
[...]
I was also thinking of using Rsync [...]
Click to expand...
The first idea is not so bad (and I think that it could be set automatic too, no need to restore all by yourself), but always remember that:
  • server A will probably contain more recent data and customers don't like losing data, even more than losing access for a few hours, so...
  • ...when server A comes back you will have to merge the data from server B to the data from server A... and this is very, very painful and slow, so...
  • ...is it better for your customers to lose access to their account for a couple days (and receive a partial refound), or to have the failover server coming up a few hours later, with old data, then wait weeks before accessing the data that went away with server A or even lose that data forever (and definitely change webhosting services)?

congkai said:
I was trying to think of the best plan to have lest down time assuming my 1st server is down.

Which is the best way? :confused:
Click to expand...
Resuming all my considerations I think that the best thing is to purchase a new server B or ask the datacenter to replace server A only in case of an incident to server A, then restore the backups... and that's all. It will take a few hours at most (if it's a good datacenter :D), that's the same time it would take to change the DNS records for a remote failover server.
Bad failover servers are worst than no failover servers at all.
 
Last edited:
Firstly THANK YOU TILO FOR YOUR LONG MESSAGE.


Different datacenter so incase network down, switch down or data center power failure.

Both center transfer at the speed of 7gb per 2 hours. (Tested!)

Traffic are free.
 
Last edited:
You are welcome. I always try to find the time to be clear about my points... and sometimes I succeed :)

Of course different datacenters is a plus for network reliability, but network failures rarely take more than a few minutes to be solved.
If I did my math correctly, 7GB in 2 hours is almost 8Mbps... which is not very much.
Anyway, as I pointed out in my last post the problem is not the speed between those datacenters, the problems are traffic (but since it's unmetered this problem is solved) and, mainly, the fact that those servers can't have the same IP address assigned.

I'll give you an example.
  • you have server A with IP 1.1.1.1 and server B (a constantly synchronized copy of server A) with IP 2.2.2.2
  • your customer example.com has DNS records "www" and "mail" that point to 1.1.1.1
  • ISPs around the world cache the record "www.example.com IN A 1.1.1.1" in their nameservers for 4 hours (default records TTL value on DA)
  • server A crashes, 1.1.1.1 is not reachable anymore
Here you have the difference:
  • you can reroute address 1.1.1.1 to server B
  • web and email services for example.com are instantly accessible for everyone again
In your case:
  • you can't reroute address 1.1.1.1 to server B
  • you have to modify "www" and "mail" records for example.com from 1.1.1.1 to 2.2.2.2
  • few people will have fast access to the services for example.com again, most people will have to wait up to 4 hours (and some ISPs cache records even for more time than the TTL value!)

If you set TTL too low, ISPs will cache those records anyway for an unpredictible time (to avoid wasting bandwidth).

In 4-6 hours you should be able to reach your datacenter's support team, ask for an hard disk (or entire server) replacement and restore all backups -- that's it, you don't need to pay another server.

Failover servers are useful when able to be deployed in less than a few minutes. Otherwise they are just wasted money, time and work because good/cheap support and good/cheap backups can do the same thing: restore services in a few hours.
 
Here's what I call poor man's redirection. It works, but it only manages redirection; you've got to keep the two servers in synchronization yourself.

Set up the zone on server A to point the domain(s) to the IP#(s) on server A, and the zone on server B to point the domain(s) to the IP#(s) on server B.

Keep both machines synchronized.

Keep a low TTL value (I generally use 600, which is ten minutes).

Here's the way DNS works: When someone tries to find your domain, their own system will ask one of their assigned caching nameserver the IP# for your domain. If the caching nameserver knows the IP#, it'll serve it, which is why I recommend a low TTL value.

But if it doesn't know the IP#, it checks DNS to find out your assigned nameservers. Those are the nameservers you can see in the whois record for your domain. Let's call them ns1.example.com and ns2.example.com (above I call them just ns1 and ns2).

What many people don't understand is which of the (generally two) nameservers it checks.

It checks both.

If the two IP#s differ (which they will following my scenario above, where each nameserver returns the IP# of the local machine), then the question is which one will it use.

The answer is, it'll use the first one it gets, and will throw away any others it gets, unread.

So, if both servers are running, some viewers will see your domain on server A, and others will see your domain on server B.

And if one server is down, it's nameserver won't be serving the IP#(s) for that server.

There are real problems with this method, especially if your sites are database driven, as they won't always be up to date.

And if your viewers spend more than the TTL time at your site, it's posible they'll move from server to server, which could cause problems with authentication sessions as well.

But for some people it works.

If it won't work for you, then a multiserver configuration becomes a lot more complex, and generally always has a single point of failure.

Jeff
 
I haven't seen any issues with ten minutes in many, many years. Commercial solutions for applying DNS to dynamic IP# assignments generally use even shorter TTLs. Examples can be found here.

Jeff
 
You must log in or register to reply here.
Back
Top