Load balanced hosting setup (5 servers)

J

jep

Verified User
Joined
Nov 12, 2003
Messages
79
Hey guys,

There are a lot of topics about load balancing, but my idea is a bit different, I think. :)

I want to make the following:

- 1 SQL-server (Dual opteron)
- 1 Fileserver (SATA raid5)
- 1 Load balancer (LVS)

So far, so good. I can define the SQL server in mysql.conf (I've done that before).

The fileserver is also easy to do. I've done this before aswell. I use NFS and all the data (home, directadmin, var) is on the remote fileserver (Except of some ip-config scripts).

The load balancer can check health, load, etcetera. :)

DNS is something we don't do on Directadmin. We have 3 external nameservers. :)

Now this is my problem:

HTTP traffic is easy to handle with this setup. The load balancer 'balances' (duh) between the 2 webservers. The only thing is that the servers (2) should both handle email (and balance the traffic aswell). Both servers will use the same email-directory's on the central fileserver, but is that possible at all?

I'm affraid that balancing my email is harder then http. :)

Does anybody have experience with this? Does it have locking problems?

Thanks for your feedback. :)
 
Kick :)

One solution can be disabling the mailserver on 1 of the balanced web/da/email machines, but I still hope there's a possibility to balance the email aswell.

If this works, I've got a fully loadbalanced setup. :)
 
I'm just curious as I have recently been investigating this same topic.

I'm assuming the fileserver will store all the files to be served from the webservers, if so wouldn't the transit of those files from the fileserver to the webserver and then to the end-client slow everything down immensely?
 
No, not at all. All big websites work like this. Think about big server setup with netapp storage machines; they are often connected by NFS to serve websites.

On Our setup, we use the SCSI raid setup as fileserver and slower IDE-machines as webservers.
 
I'm not doing the research now to find out how IMAP and/or POP manage file locking.

But I thought I should point out that NFS doesn't do locking well at all.

So that's the defining issue.

Jeff
 
NFS & POP/IMAP

I am using dovecot for my mail server on the front end server. It connects to an NFS share. I changed /etc/dovecot.conf to have these settings:

lock_method = dotlock
mmap_disable = yes

That allowed better handling of file locking in dovecot.
 
Locking using files as semaphores generally works well on all file systems.

I just don't know how the systems included with DA do it and I'm not taking the time to look it up; I leave that as an exercise for the admin trying to implement NFS :) .

Jeff
 
I have not tried with two front end servers yet, but it works fine with just one. I origainlly ran into file locking issues with dovecot, but found this works just fine.

The only issue I have found is that quotas and the associated disk space reporting do not work on NFS. If anyone has an idea, please let me know and we can try.
 
I have the same issue as Greg. I'm running 2 DA front end and a NFS server connecting them with GE. Performance is fine.

- /home /etc/virtual are mounted to the NFS
- there is no file locking issue, I use dovecot/Maildir format
- DNS is handled by DA's clustering feature
- User management will be handled at one of the DA frontend (1st DA) and I write scripts to replicate from the 1st frontend to the 2nd frontend.

The only issue I have is disk space and quota is not working.
Any idea how to make quota works? :)
 
uhm guys, are you serving multiple websites with this or... ? Cus what happens when you have websites that write to files... Eg, count stuff that way... Your webservercluster will make a mess of that...

email could well be done by a single seperate machine, just put in a fallback with it and its ready.

mysql is better hosted on a dedicated machine but you already covered that. Is that realy as easy as just modifing the my.cnf ? Eg. pma still works from your DA host(s) (different machines then the sql server)?

about the quota's, arent they stored in a /.quota kind of file ? in that case you could create a simple bash script to read that quota file every X minutes and store it at a location viewable by your DA host(s). The DA host(s) will also run a script every X minutes to updates its own quota files ( that arent used anyway since everything happens on other machines) with the ones from the webcluster.
 
Last edited:
I think that quota would have to be an OS issue, rather than a DA issue; this might not be the best place to discuss it.

DA is only going to be able to count used space on what looks like a locally mounted drive. That said, based on your post I don't see why it wouldn't see space on /home.

Jeff
 
Have there been any changes on this issue since 2007?

I couldn't find a satisfactory result in the subject, I hope it will be next :)
 
You must log in or register to reply here.
Back
Top