PHP-FPM 7.4: 503 Service Unavailable

Y

Yukka

Verified User
Joined
May 29, 2018
Messages
4
Hello,

we have some weird behavior with PHP 7.4 which for already a few months prevents us from using it.

When it's used for a longer time (sometimes couple of minutes, sometimes a few hours) it's starts to crash with 503 error.
Once one script crashes, it won't work until we reload php-fpm. At the same time other scripts still work.
Interestingly, when we don't use 7.4 for longer time and then, (most often after updating 7.4.* to most recent version) we want to check if problem still exists - it crashes from the first request. One reload and it works like a charm...for some time.

So it seems like some buffer/memory/whatever fills up and won't free until php-fpm reload.

Logs are rather not helpful:
[proxy_fcgi:error] [pid 4241] [client 202.172.253.9:60170] AH01067: Failed to read FastCGI header
[proxy_fcgi:error] [pid 4241] (104)Connection reset by peer: [client 202.172.253.9:60170] AH01075: Error dispatching request to :
[proxy_fcgi:error] [pid 13136] [client 202.172.253.9:61410] AH01067: Failed to read FastCGI header
[proxy_fcgi:error] [pid 13136] (104)Connection reset by peer: [client 202.172.253.9:61410] AH01075: Error dispatching request to :
[proxy_fcgi:error] [pid 12049] [client 202.172.253.9:62679] AH01067: Failed to read FastCGI header
[proxy_fcgi:error] [pid 12049] (104)Connection reset by peer: [client 202.172.253.9:62679] AH01075: Error dispatching request to :
[proxy_fcgi:error] [pid 12413] [client 202.172.253.9:62693] AH01067: Failed to read FastCGI header
[proxy_fcgi:error] [pid 12413] (104)Connection reset by peer: [client 202.172.253.9:62693] AH01075: Error dispatching request to :
[proxy_fcgi:error] [pid 9291] [client 202.172.253.9:59183] AH01067: Failed to read FastCGI header
[proxy_fcgi:error] [pid 9291] (104)Connection reset by peer: [client 202.172.253.9:59183] AH01075: Error dispatching request to :
[proxy_fcgi:error] [pid 9077] [client 202.172.253.9:59260] AH01067: Failed to read FastCGI header
Click to expand...
kernel: [3406106.433105] php-fpm74[17563]: segfault at 0 ip 000055cb84a0655b sp 00007ffec6c76ca0 error 6 in php-fpm74[55cb845ca000+454000]
kernel: [3406106.433126] Code: 48 8b 7f 08 8b 47 04 25 10 fc ff ff 83 f8 10 74 3c 49 83 c4 10 83 ed 01 0f 84 e7 c9 ff ff 41 80 7c 24 09 00 74 eb 49 8b 3c 24 <83> 2f 01 75 c5 41 c7 44 24 08 01 00 00 00 e8 82 d3 f7 ff eb d2 49
kernel: [3406148.409564] php-fpm74[18467]: segfault at 0 ip 000055cb84a0655b sp 00007ffec6c76ca0 error 6 in php-fpm74[55cb845ca000+454000]
kernel: [3406148.409583] Code: 48 8b 7f 08 8b 47 04 25 10 fc ff ff 83 f8 10 74 3c 49 83 c4 10 83 ed 01 0f 84 e7 c9 ff ff 41 80 7c 24 09 00 74 eb 49 8b 3c 24 <83> 2f 01 75 c5 41 c7 44 24 08 01 00 00 00 e8 82 d3 f7 ff eb d2 49
kernel: [3406185.695466] php-fpm74[19253]: segfault at 0 ip 000055cb84a0655b sp 00007ffec6c76ca0 error 6 in php-fpm74[55cb845ca000+454000]
kernel: [3406185.695476] Code: 48 8b 7f 08 8b 47 04 25 10 fc ff ff 83 f8 10 74 3c 49 83 c4 10 83 ed 01 0f 84 e7 c9 ff ff 41 80 7c 24 09 00 74 eb 49 8b 3c 24 <83> 2f 01 75 c5 41 c7 44 24 08 01 00 00 00 e8 82 d3 f7 ff eb d2 49
kernel: [3406363.723260] php-fpm74[24386]: segfault at 0 ip 000055cb84a0655b sp 00007ffec6c76ca0 error 6 in php-fpm74[55cb845ca000+454000]
kernel: [3406363.723271] Code: 48 8b 7f 08 8b 47 04 25 10 fc ff ff 83 f8 10 74 3c 49 83 c4 10 83 ed 01 0f 84 e7 c9 ff ff 41 80 7c 24 09 00 74 eb 49 8b 3c 24 <83> 2f 01 75 c5 41 c7 44 24 08 01 00 00 00 e8 82 d3 f7 ff eb d2 49
kernel: [3406364.808019] php-fpm74[24388]: segfault at 0 ip 000055cb84a0655b sp 00007ffec6c76ca0 error 6 in php-fpm74[55cb845ca000+454000]
kernel: [3406364.808029] Code: 48 8b 7f 08 8b 47 04 25 10 fc ff ff 83 f8 10 74 3c 49 83 c4 10 83 ed 01 0f 84 e7 c9 ff ff 41 80 7c 24 09 00 74 eb 49 8b 3c 24 <83> 2f 01 75 c5 41 c7 44 24 08 01 00 00 00 e8 82 d3 f7 ff eb d2 49
Click to expand...

We even updated Debian 9 to Debian 10 hoping it will resolve the issue, but it's exactly the same. On Debian 9 7.4 was installed as additional PHP, on Debian 10 it's a main PHP, but we still run on additional 7.3.

Currently our config is:
Apache 2.4.46
Php 7.4.12
Php 7.3.24 (php2)

With previous migrations from 5.6 -> 7.0 -> 7.1 -> 7.2 -> 7.3 we did not have problems like this.

When we run the same scripts on local environment on PHP 7.4 there are no problems.

Has anybody have some idea how to fix it? :) Or even how to trace the reason that could be behind it?
 
Some off the top thoughts

So when you upgraded to debain 10 did you rerun the preinstall packages?

Code: 
apt-get update
apt-get install wget gcc g++ make flex bison openssl libssl-dev perl perl-base perl-modules libperl-dev libperl4-corelibs-perl libwww-perl libaio1 libaio-dev \
zlib1g zlib1g-dev libcap-dev cron bzip2 zip automake autoconf libtool cmake pkg-config python libdb-dev libsasl2-dev \
libncurses5 libncurses5-dev libsystemd-dev bind9 dnsutils quota patch logrotate rsyslog libc6-dev libexpat1-dev \
libcrypt-openssl-rsa-perl libnuma-dev libnuma1

did you run
Code: 
apt autoremove
to clean up old orphaned packages

Did you change your license in the customer portal to Debian 10?

Also, did you run
Code: 
cd /usr/local/directadmin/custombuild
./build set_fastest
./build clean
./build update
./build all
./build rewrite_confs

For starters.
 
Hello,

thank you for ideas. Yes, it's all been done. Moreover, the same problem was before updating to Debian 10, so update itself is not the cause of problems.
 
What is the Server cpu and memory look like

what does htop or free -m

What php applications are you using?

If yous witch back to 7.3 does the issue go away?
 
18 CPUs & 24 GB RAM.

It's a tailor-made B2B platform.

Yes, on 7.3 we don't have any problems. 7.3 is default version for the platform. I've created subdomain pointing to the same dir like main domain, but with php 7.4 set. So the environments are exactly the same. I was thinking that maybe this is the problem - that two php versions are running the same scripts? Maybe something with opcache?
 
When you compiled 7.3 did you add any custom compile flags on the (Customize Compilation) tab? if yes did you add them to 7.4?

You might have the developer look here.

www.php.net

PHP: Migrating from PHP 7.3.x to PHP 7.4.x - Manual

PHP is a popular general-purpose scripting language that powers everything from your blog to the most popular websites in the world.
www.php.net www.php.net

www.php.net

PHP: Backward Incompatible Changes - Manual

PHP is a popular general-purpose scripting language that powers everything from your blog to the most popular websites in the world.
www.php.net www.php.net

www.php.net

PHP: Removed Extensions - Manual

PHP is a popular general-purpose scripting language that powers everything from your blog to the most popular websites in the world.
www.php.net www.php.net
 
Yes, both 7.3 and 7.4 have the same settings.
But it works fine with 7.4 on our staging/local environment.
 
Code: 
ulimit -a
I meant a sorry

Code: 
cat /etc/security/limits.conf
Maybe you need to check ulimits
 
Last edited:
You must log in or register to reply here.
Back
Top