How to effectively handle spikes of requests?

[kristian]

Every now and then, a website "goes viral" and receives a high number of legitimate requests. Since a typical DirectAdmin server setup is single-server only, there's nowhere to spread the load, and the single server must deal with all the incoming requests all on its own.

My question is how do people deal with spikes in number of legitimate requests? How do you set thresholds for max apache workers, or php-fpm processes? How do you scale your servers in terms of CPU/RAM vs customers/domains. Do you implement any techniques/technologies on-server or pre-server (e.g. cloudflare) for caching purposes? Do you tweak kernel parameters (sysctl) to better handle incoming connections? Do you do anything else that I can't think of?

 

[k1l0b1t]

In case of wordpress sites, I've noticed adding caching (with a plugin like wp super cache) can help.
It can both act as a simple cache (which improves general performance), but you can also make some pages super-supercached, they're essentialy turned into a static page. on a 2c server it can boost requests from ~6/sec (without), to 80/sec with simple caching, and up to 500 on the pages that are super-cached.

 

[cjd]

Really depends on what the sites are, for what kinds of optimizations that may be needed.

I host a bunch of WordPress sites for conferences that can get hit pretty hard. Those server are running Apache w/Nginx cache. I have adjusted the php-fpm for dynamic with more min and spare servers (seems to work better for my case). I use 'W3 Total Cache' for a WordPress cache. Also disable the web cron and setup wp-cli to run the cron. Keep plugins to a minimum, designers get carried away all the time and the complain that it's slow (when they install 50 bloated plugins and don't really use any of them, or just a single function from one that a couple lines of css could do).

And make sure MySQL/MariaDB are optimized for the databases they have (MySQLTuner comes in handy https://github.com/major/MySQLTuner-perl ).

Also I don't overload my servers, and give them plenty of RAM. Also monitor them to make sure they are preforming well.

If a site is possibly known that it will go viral, the better cached/static it is the better. Using a CDN can also help with offloading static content. Cloudflare can be beneficial for cache and DDOS protection.

As for system level tweaks, good to make sure your swappiness is configured, I use 1%.

Besides that most of the defaults have been fine. I could see increasing apache workers may be necessary at some point, as long as the system can handle the additional requests.

All my installs use Debian (Almost all upgraded to 11 now, just a couple left on 9). Most of my servers are still spinning rust, been a slow upgrade to SSD/NVME.

I do use Proxmox as a hypervisor on most servers, don't normally run more than a couple VMs on one physical server for production, makes it easy to backup/restore/migrate them quickly to a new host, and I don't really see much degradation in performance.

I have been doing some testing with different linux kernels to see if they make any difference, so far pretty minimal.

 

[Zhenyapan]

it's easier to keep enough free resources with more powerful server.
for example earlier we have a lot of 4c8t 32ram servers for hosting and each can serve ~100 websites with different engines, each spike can drop whole server. In normal each server was loaded at 25%. But when we move it to 28c56t 256ram - we can replace not 6 servers (according to core count ratio) but 10 - more cores allows to pass requests faster, and server still loaded an 25%, but earlier we have 3 cores in reserve in each server, and now 21 cores - it's enough to handle any spikes. Also it's cheaper. Also we don't need 10 licenses of each product (DA, Antivirus, Cloudlinux etc.) but only one - so we can use better hardware nvme, faster ram, etc..
----
if you want to tune same server - check for caching and optimizing websites/plugins/themes/mysql. Maybe move from apache to nginx+apache to nginx+php-fpm to openlitespeed. Change mysql engine, from myisam to innodb only or something else.

 

[brinkie]

Every now and then, a website "goes viral" and receives a high number of legitimate requests.

Did you consider using a service like Cloudflare or another CDN network? Esepcially if this happens a lot it might be well worth the money? And of course adding popular websites to Cloudflare even using the free plan can save you from a lot of trouble. I use it for several clients that have very popular websites and works perfect.