PHP-FPM nightmares

Needing many children processes would mean usually either 1 of the 2, or a combination I think:
1) simply a lot of traffic
2) some pages, or the base system (then it would mean all pages), use a lot of calculations or inefficient database queries etc.. something that is causing 1 request to take long/a lot of resources. Thus the amount of requests that can be handled in a certain period of time, is limited by the inefficiency, thus needing more children to handle the needed requests.
 
wtptrs said:
You should increase max_children according to how much RAM your server has. One php-fpm process usually uses about 50-80MB of RAM.
Click to expand...
I used the ps_mem.py and the outcome was
Code: 
 16.7 MiB +   1.6 MiB =  18.2 MiB       php-fpm74
600.0 MiB +  77.7 MiB = 677.7 MiB       php-fpm73 (41)
So, by my calculations of (Total RAM - Memory used for Linux, DB, etc.) / process size) for max_children, would be 46 (eg. (32768-2000)/678 - albeit, the 2gb is a random number)
 
Yes, of course it's better to actually properly calculate :) but this should fix the issue you described earlier
 
Because it increases the memory usage and the benefits are questionable for shared hosting. Just test it and see how it will go.
 
Just notice this in php-fpm log
Code: 
[18-Nov-2020 00:11:01] NOTICE: Reloading in progress ...
I guess a reload is different then a restart I suppose.
 
Yes max_children is important. If this is a dedicated high traffic site 100 is better.

Here is more info on how to tune it and how it works

www.php.net

PHP: Configuration - Manual

PHP is a popular general-purpose scripting language that powers everything from your blog to the most popular websites in the world.
www.php.net www.php.net
pm string
Choose how the process manager will control the number of child processes. Possible values: static, ondemand, dynamic. This option is mandatory.

static - the number of child processes is fixed (pm.max_children).

ondemand - the processes spawn on demand (when requested, as opposed to dynamic, where pm.start_servers are started when the service is started.

dynamic - the number of child processes is set dynamically based on the following directives: pm.max_children, pm.start_servers, pm.min_spare_servers, pm.max_spare_servers.

pm.max_children int
The number of child processes to be created when pm is set to static and the maximum number of child processes to be created when pm is set to dynamic. This option is mandatory.

This option sets the limit on the number of simultaneous requests that will be served. Equivalent to the ApacheMaxClients directive with mpm_prefork and to the PHP_FCGI_CHILDREN environment variable in the original PHP FastCGI.

pm.start_servers int
The number of child processes created on startup. Used only when pm is set to dynamic. Default Value: min_spare_servers + (max_spare_servers - min_spare_servers) / 2.

pm.min_spare_servers int
The desired minimum number of idle server processes. Used only when pm is set to dynamic. Also mandatory in this case.

pm.max_spare_servers int
The desired maximum number of idle server processes. Used only when pm is set to dynamic. Also mandatory in this case.

pm.process_idle_timeout mixed
The number of seconds after which an idle process will be killed. Used only when pm is set to ondemand. Available units: s(econds)(default), m(inutes), h(ours), or d(ays). Default value: 10s.

pm.max_requests int
The number of requests each child process should execute before respawning. This can be useful to work around memory leaks in 3rd party libraries. For endless request proce
Click to expand...
Ok so lets be a bit more fair to the entire box
You have 32 gigs of RAM and the server is only this site correct? if yes
Let's say we have these things that need memory
OS
Webserver
Mariadb
PHP FPM
and Other stuff and overhead
so we divide 32 by 5

each major item can have 6.4 gigs of ram

Now look at PHP-fpm

serverfault.com

Php-Fpm 7 server reached pm.max_children

I am receiving this error message when I run an intensive job in Wordpress: [pool www] server reached pm.max_children setting (5), consider raising it Using Php-fpm 7 + Nginx on 2GB RAM Server. ...
serverfault.com serverfault.com
you can use these commands

This gets average php memory per request

Code: 
ps --no-headers -o "rss,cmd" -C php-fpm74 | awk '{ sum+=$1 } END { printf ("%d%s\n", sum/NR/1024,"M") }'
change php-fpm74 number as needed above

take this number and plug it into amount of Ram you want to give to PHP FPM in the above 6656MB / number from cmd = max children

This will give you a better overall coverage. Because you will need to tune MariaDB next
 
Last edited:
wattie said:
Because it increases the memory usage and the benefits are questionable for shared hosting. Just test it and see how it will go.
Click to expand...
Can you tell where you get that info? Because reading around, even with hosting company's, they say it's giving a nice performance winning.
Until now I didn't see that benefits would be questionable for shared hosting. Even my private forum is running faster with opache enabled.
 
Richard G said:
Can you tell where you get that info? Because reading around, even with hosting company's, they say it's giving a nice performance winning.
Until now I didn't see that benefits would be questionable for shared hosting. Even my private forum is running faster with opache enabled.
Click to expand...
It's not questionable for us, makes a big difference on a lot of our websites.
 
Going off-topic a bit, opcache is a burden for developers - unless I'm using opcache wrong - I find it takes ages to find the change files. Although, I assume it's the same when you use Cloudflare's proxy......
 
This is getting ridiculous now....... Getting this....... and also mod_security hits galore.... disabling it seems to be better, but obviously with a WP site, hell no!

Server has been fine for 2 solid months then it decides to boob up.

I'm running out of patience, tempted to just lose the client.
 

Attachments

  • 126163071_10160219221289256_4496194830311830092_o.jpg
    126163071_10160219221289256_4496194830311830092_o.jpg
    57.5 KB · Views: 8
Could be a cloudflare issue. The post below is just from sunday.

I'm having Error 524

Hello, I have 3 websites with DNS on cloudflare and installed on DA VPS, but I only successfully access 2 domains, and one error 524. I tried installing many times and reset DNS, but unsuccessful.
forum.directadmin.com forum.directadmin.com

community.cloudflare.com

Community Tip - Fixing Error 524: A timeout occurred

Error Try the suggestions in this Community Tip to help you fix Error 524: A timeout occurred. Background Error 524 error indicates that Cloudflare made a successful TCP connection to the origin web server, but the origin did not reply with an HTTP response before the connection timed out...
community.cloudflare.com community.cloudflare.com
 
Do I need to whitelist Cloudflare's proxy IP's?

Edit: I just have, just in case.

Edit 2: Found this, the last post is interesting https://community.cloudflare.com/t/524-error-on-my-website/121183

Edit 3:
Cloudflare will typically wait for a HTTP response from your server for 100 seconds. If no response(<8KB) is sent by your server in that time, Cloudflare will close the connection and serve a 524 error page.
Click to expand...
Or it could be this..... 100+ to serve a page is ludicrous, if it is this problem
 
Last edited:
Well, I guess it wasn't enough
Code: 
[19-Nov-2020 05:58:07] WARNING: [pool xxx] server reached max_children setting (46), consider raising it
Now changed to 100 ?‍♂️

Now the user's fpm.conf is
Code: 
pm = ondemand
pm.max_children = 100
pm.process_idle_timeout = 20
pm.max_requests = 500
Maybe the timeout needs lowering........

<Lucky I'm already bald, or I would be by now>
 
You must log in or register to reply here.
Back
Top