Highly-Scalable Architecture

[mrfileio]

I have a requirement to serve up to 50,000 users with one subdomain per
user, for example, user1.bigdomain.com, user2.bigdomain.com, etc.

Though there would be 50K users/domains supported, only about 10%,
or 5K users/domains, would be "active" users. These "active" users would create web sites and a small percentage of them would use mysql in their web sites.

Taking disk requirements out of the picture, could a single, really large, i.e.
quad core 3GHz, 32GB RAM computer handle this?

If not, what are the strategies to subdivide these users/domains across
multiple servers?

Is there a tried and tested approach to moving the mysql db to its own server and horizontally scaling the users across multiple servers which is transparent to the end users?

An important point is that there will be no email interration for these users. This is for hosting web sites only.


Thanks, Chris

 

[DirectAdmin Support]

Hello,

I believe that with that many virtualhosts, apache would takes ages to boot up with the config system we use with DirectAdmin.

As long as this is a specific setup and you don't plan on doing hosting for other domains in a normal format (where you'd want the DA automation of apache configs).. then I would recommend using apache's mass virtual hosting option:
http://httpd.apache.org/docs/2.0/vhosts/mass.html

What it does is, instead of a VH for each subdomain, it would simply accept the subdomain name via the "Host:" part of the request, using it as a variable, and simply go to that location on disk to look for it. This means that with 50,000 subdomains, you'd only need 1 fairly simple VirtualHost, which would allow apache to boot up very quickly, and it would also run far quicker since there wouldn't be a giant lookup table to match the Host value. It would just use the subdomain name variable.

Note that Mass virtual hosting is not a stock option in DA, but with all of the custom templates, and pre/post scripts, is it entirely possible to setup in a customized way. Also keep in mind that each subdomain would have the same apache options, so you couldn't really customize things on a per-VH level since all 50,000 would be controlled by just the 1 Mass VH.

As for hardware, you will really have no way of knowing how many users you can pack onto the box. This is because there are too many variables that each new user will provide. For example, you could in theory have just 1 subdomain that is so popular it uses all of your resources. On the other hand, if they're all inactive, then in theory the limit could be very very high.

The biggest hurdle would be if you're allowing databases for these subdomains, and/or php/cgi-bin access. Scripts and databases are the highest users of cpu/ram resources, as compared to just static html/file downloads.

Related:
http://help.directadmin.com/item.php?id=233

John