PHP-FPM56 problem with memory

M

Mattpl

Verified User
Joined
Jun 28, 2017
Messages
84
Location
ZS
Hi! This happend 2 times, scripts (and website) can't load because have error (code 3004, read timeout). I have check my sevices and phpfpm56 have a lot of pids and memory usage - 3.3GB. When i restart it and httpd I can load script and website works fine.

Could you tell me how config phpfpm for better memory usage.
 
Hello,

There are several possible reasons on that:

1. Malware
2. Brute-force attack
3. Other unwanted scenario
4. MySQL tables crashed

Check server status in Apache, Apache logs for repeated POST requests.
 
Everyday I have info about brutforce attack's (mails from csf).
httpd status
● httpd.service - The Apache HTTP Server
Loaded: loaded (/etc/systemd/system/httpd.service; enabled)
Active: active (running) since Thu 2017-10-12 17:41:39 CEST; 3h 21min ago
Process: 16898 ExecStop=/bin/kill -WINCH ${MAINPID} (code=exited, status=0/SUCCESS)
Main PID: 16904 (httpd)
Status: "Total requests: 20611; Idle/Busy workers 93/6;Requests/sec: 1.71; Bytes served/sec: 32KB/sec"
CGroup: /system.slice/httpd.service
├─12535 /usr/sbin/httpd -DFOREGROUND
└─16904 /usr/sbin/httpd -DFOREGROUND
Click to expand...
Service stopped at 17.30
this is apache error log
[Thu Oct 12 17:41:32.645708 2017] [mpm_event:notice] [pid 2514:tid 140232315025216] AH00492: caught SIGWINCH, shutting down gracefully
[Thu Oct 12 17:41:38.004242 2017] [ssl:warn] [pid 16904:tid 140604141729600] AH01909: localhost:443:0 server certificate does NOT include an ID which matches the server name
[Thu Oct 12 17:41:38.004530 2017] [ssl:warn] [pid 16904:tid 140604141729600] AH01909: www.example.com:443:0 server certificate does NOT include an ID which matches the server name
[Thu Oct 12 17:41:38.004592 2017] [suexec:notice] [pid 16904:tid 140604141729600] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Thu Oct 12 17:41:38.004602 2017] [:notice] [pid 16904:tid 140604141729600] ModSecurity for Apache/2.9.2 (http://www.modsecurity.org/) configured.
[Thu Oct 12 17:41:38.004605 2017] [:notice] [pid 16904:tid 140604141729600] ModSecurity: APR compiled version="1.6.2"; loaded version="1.6.2"
[Thu Oct 12 17:41:38.004608 2017] [:notice] [pid 16904:tid 140604141729600] ModSecurity: PCRE compiled version="8.20 "; loaded version="8.20 2011-10-21"
[Thu Oct 12 17:41:38.004610 2017] [:notice] [pid 16904:tid 140604141729600] ModSecurity: LIBXML compiled version="2.9.4"
[Thu Oct 12 17:41:38.004612 2017] [:notice] [pid 16904:tid 140604141729600] ModSecurity: Status engine is currently disabled, enable it by set SecStatusEngine to On.
[Thu Oct 12 17:41:39.002905 2017] [ssl:warn] [pid 16904:tid 140604141729600] AH01909: localhost:443:0 server certificate does NOT include an ID which matches the server name
[Thu Oct 12 17:41:39.003205 2017] [ssl:warn] [pid 16904:tid 140604141729600] AH01909: www.example.com:443:0 server certificate does NOT include an ID which matches the server name
[Thu Oct 12 17:41:39.003508 2017] [lbmethod_heartbeat:notice] [pid 16904:tid 140604141729600] AH02282: No slotmem from mod_heartmonitor
[Thu Oct 12 17:41:39.005246 2017] [mpm_event:notice] [pid 16904:tid 140604141729600] AH00489: Apache/2.4.27 (Unix) OpenSSL/1.0.1t configured -- resuming normal operations
[Thu Oct 12 17:41:39.005275 2017] [core:notice] [pid 16904:tid 140604141729600] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
Click to expand...




Want install mode_security Comodo from CB 2.0
Can't locate Comodo/CWAF/Main.pm in @INC (you may need to install the Comodo::CWAF::Main module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.20.2 /usr/local/share/perl/5.20.2 /usr/lib/x86_64-linux-gnu/perl5/5.20 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.20 /usr/share/perl/5.20 /usr/local/lib/site_perl .) at /usr/local/cwaf/scripts/updater.pl line 12.
BEGIN failed--compilation aborted at /usr/local/cwaf/scripts/updater.pl line 12.
Installation of ModSecurity Rule Set has been finished.
Done!
Click to expand...
 
Last edited:
I have a lot of request from
106.51.37.22 - - [13/Oct/2017:01:22:01 +0200] "HEAD http://193.107.88.114:80/mysql/pMA/ HTTP/1.1" 404 181 "-" "Mozilla/5.0 Jorgee"
106.51.37.22 - - [13/Oct/2017:01:22:02 +0200] "HEAD http://193.107.88.114:80/sql/phpmanager/ HTTP/1.1" 404 181 "-" "Mozilla/5.0 Jorgee"
106.51.37.22 - - [13/Oct/2017:01:22:02 +0200] "HEAD http://193.107.88.114:80/sql/php-myadmin/ HTTP/1.1" 404 181 "-" "Mozilla/5.0 Jorgee"
106.51.37.22 - - [13/Oct/2017:01:22:02 +0200] "HEAD http://193.107.88.114:80/sql/phpmy-admin/ HTTP/1.1" 404 181 "-" "Mozilla/5.0 Jorgee"
106.51.37.22 - - [13/Oct/2017:01:22:02 +0200] "HEAD http://193.107.88.114:80/sql/sql/ HTTP/1.1" 404 181 "-" "Mozilla/5.0 Jorgee"
Click to expand...

23.28.123.161 - - [12/Oct/2017:20:11:17 +0200] "HEAD http://193.107.88.114:80/pma2016/ HTTP/1.1" 404 180 "-" "Mozilla/5.0 Jorgee"
23.28.123.161 - - [12/Oct/2017:20:11:17 +0200] "HEAD http://193.107.88.114:80/pma2017/ HTTP/1.1" 404 180 "-" "Mozilla/5.0 Jorgee"
23.28.123.161 - - [12/Oct/2017:20:11:18 +0200] "HEAD http://193.107.88.114:80/pma2018/ HTTP/1.1" 404 180 "-" "Mozilla/5.0 Jorgee"
23.28.123.161 - - [12/Oct/2017:20:11:18 +0200] "HEAD http://193.107.88.114:80/phpmyadmin2011/ HTTP/1.1" 404 180 "-" "Mozilla/5.0 Jorgee"
23.28.123.161 - - [12/Oct/2017:20:11:18 +0200] "HEAD http://193.107.88.114:80/phpmyadmin2012/ HTTP/1.1" 404 180 "-" "Mozilla/5.0 Jorgee"
23.28.123.161 - - [12/Oct/2017:20:11:18 +0200] "HEAD http://193.107.88.114:80/phpmyadmin2013/ HTTP/1.1" 404 180 "-" "Mozilla/5.0 Jorgee"
23.28.123.161 - - [12/Oct/2017:20:11:18 +0200] "HEAD http://193.107.88.114:80/phpmyadmin2014/ HTTP/1.1" 404 180 "-" "Mozilla/5.0 Jorgee"
23.28.123.161 - - [12/Oct/2017:20:11:18 +0200] "HEAD http://193.107.88.114:80/phpmyadmin2015/ HTTP/1.1" 404 145 "-" "Mozilla/5.0 Jorgee"
23.28.123.161 - - [12/Oct/2017:20:11:19 +0200] "HEAD http://193.107.88.114:80/phpmyadmin2018/ HTTP/1.1" 404 182 "-" "Mozilla/5.0 Jorgee"
23.28.123.161 - - [12/Oct/2017:20:11:19 +0200] "HEAD http://193.107.88.114:80/phpmanager/ HTTP/1.1" 404 182 "-" "Mozilla/5.0 Jorgee
Click to expand...

178.15.98.24 - - [12/Oct/2017:19:31:10 +0200] "HEAD http://193.107.88.114:80/PMA2012/ HTTP/1.1" 404 181 "-" "Mozilla/5.0 Jorgee"
178.15.98.24 - - [12/Oct/2017:19:31:10 +0200] "HEAD http://193.107.88.114:80/PMA2013/ HTTP/1.1" 404 181 "-" "Mozilla/5.0 Jorgee"
178.15.98.24 - - [12/Oct/2017:19:31:10 +0200] "HEAD http://193.107.88.114:80/PMA2014/ HTTP/1.1" 404 181 "-" "Mozilla/5.0 Jorgee"
178.15.98.24 - - [12/Oct/2017:19:31:10 +0200] "HEAD http://193.107.88.114:80/PMA2015/ HTTP/1.1" 404 181 "-" "Mozilla/5.0 Jorgee"
178.15.98.24 - - [12/Oct/2017:19:31:10 +0200] "HEAD http://193.107.88.114:80/PMA2016/ HTTP/1.1" 404 181 "-" "Mozilla/5.0 Jorgee"
178.15.98.24 - - [12/Oct/2017:19:31:11 +0200] "HEAD http://193.107.88.114:80/PMA2017/ HTTP/1.1" 404 181 "-" "Mozilla/5.0 Jorgee"
Click to expand...

37.145.102.98 - - [12/Oct/2017:19:11:07 +0200] "HEAD http://193.107.88.114:80/phpMyAdmin4/ HTTP/1.1" 404 181 "-" "Mozilla/5.0 Jorgee"
37.145.102.98 - - [12/Oct/2017:19:11:08 +0200] "HEAD http://193.107.88.114:80/phpMyAdmin-3/ HTTP/1.1" 404 181 "-" "Mozilla/5.0 Jorgee"
37.145.102.98 - - [12/Oct/2017:19:11:08 +0200] "HEAD http://193.107.88.114:80/php-my-admin/ HTTP/1.1" 404 181 "-" "Mozilla/5.0 Jorgee"
37.145.102.98 - - [12/Oct/2017:19:11:08 +0200] "HEAD http://193.107.88.114:80/PMA2011/ HTTP/1.1" 404 181 "-" "Mozilla/5.0 Jorgee"
Click to expand...

89.64.35.216 - - [12/Oct/2017:17:43:02 +0200] "GET /phpmyadmin/js/messages.php?v=4.7.4 HTTP/1.1" 304 392 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"
89.64.35.216 - - [12/Oct/2017:17:43:02 +0200] "GET /phpmyadmin/js/get_scripts.js.php?scripts%5B%5D=codemirror/addon/hint/show-hint.js&scripts%5B%5D=codemirror/addon/hint/sql-hint.js&scripts%5B%5D=codemirror/addon/lint/lint.js&scripts%5B%5D=codemirror/addon/lint/sql-lint.js&scripts%5B%5D=console.js&v=4.7.4 HTTP/1.1" 200 20007 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"
89.64.35.216 - - [12/Oct/2017:17:43:03 +0200] "GET /phpmyadmin/themes/pmahomme/css/printview.css?v=4.7.4 HTTP/1.1" 200 1386 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"
89.64.35.216 - - [12/Oct/2017:17:43:02 +0200] "GET /phpmyadmin/js/get_scripts.js.php?scripts%5B%5D=navigation.js&scripts%5B%5D=indexes
Click to expand...

ban this ip's?
 
The log-lines show the requests which hardly could cause a higher-load on tour server, even with 6+ req/s. The 404 error page which is shown apon the request is static.

Apache logs are not limited with /var/log/httpd/access_log and error_log, there are logs in /var/log/httpd/domains/ and rotated in domain's dir of user homedirs.

Check this: https://help.directadmin.com/?query=server-status
 
I have only 1 website on this server.
Check this logs for my domain and before crash and after I have a lot of GET request (looks like ddos?? too much request; something scan website?)

95.85.80.68 - - [13/Oct/2017:03:53:34 +0200] "GET /ogloszenie?SearchForm%5Bpracodawca%5D=201 HTTP/1.1" 200 33193 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
24.212.48.181 - - [13/Oct/2017:03:53:32 +0200] "GET /praca/podlaskie HTTP/1.1" 200 29813 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
142.165.226.197 - - [13/Oct/2017:03:53:34 +0200] "GET /ogloszenie?SearchForm%5Bpracodawca%5D=6 HTTP/1.1" 200 28150 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
185.53.44.94 - - [13/Oct/2017:03:53:23 +0200] "GET /ogloszenie/show/73366,specjalista-ds-sprzedazy HTTP/1.1" 200 72815 "-" "Mozilla/5.0 (compatible; XoviBot/2.0; +http://www.xovibot.net/)"
146.185.206.68 - - [13/Oct/2017:03:53:31 +0200] "GET /porady-i-wzory-cv?page=2 HTTP/1.1" 200 30397 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
191.101.55.48 - - [13/Oct/2017:03:53:34 +0200] "GET /wiadomosc/62,urlop-bez-telefonu-jest-mozliwy- HTTP/1.1" 200 32917 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
212.35.179.146 - - [13/Oct/2017:03:53:34 +0200] "GET /ogloszenie/region/2,lubelskie/strona/10 HTTP/1.1" 200 34341 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
191.101.54.232 - - [13/Oct/2017:03:53:30 +0200] "GET /ogloszenie?SearchForm%5Bpracodawca%5D=6 HTTP/1.1" 200 28147 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
126.82.153.89 - - [13/Oct/2017:03:53:35 +0200] "GET /praca/podkarpackie HTTP/1.1" 200 137 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
61.7.170.226 - - [13/Oct/2017:03:53:35 +0200] "GET /ogloszenie?SearchForm%5Bpracodawca%5D=201 HTTP/1.1" 200 29553 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
153.213.226.84 - - [13/Oct/2017:03:53:36 +0200] "GET /ogloszenie/show/115321,sprzedawca HTTP/1.1" 200 137 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
178.159.97.124 - - [13/Oct/2017:03:53:35 +0200] "GET /praca/podlaskie HTTP/1.1" 200 29806 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
175.179.14.14 - - [13/Oct/2017:03:53:36 +0200] "GET /wiadomosc/103,co-grozi-za-nieudzielenie-pomocy HTTP/1.1" 200 31349 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
61.72.95.132 - - [13/Oct/2017:03:53:35 +0200] "GET /ogloszenie/kategoria/158,obsluga-klienta HTTP/1.1" 200 137 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
46.148.127.88 - - [13/Oct/2017:03:53:36 +0200] "GET /ogloszenie/kategoria/158,obsluga-klienta HTTP/1.1" 200 29581 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
185.14.195.159 - - [13/Oct/2017:03:53:37 +0200] "GET /ogloszenie/show/115378,lakiernik-samochodowy HTTP/1.1" 200 28854 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
91.200.82.172 - - [13/Oct/2017:03:53:36 +0200] "GET /praca/podlaskie HTTP/1.1" 200 33179 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
163.47.115.25 - - [13/Oct/2017:03:53:37 +0200] "GET /ogloszenie?SearchForm%5Bpracodawca%5D=201 HTTP/1.1" 200 29557 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
46.248.167.156 - - [13/Oct/2017:03:53:37 +0200] "GET /ogloszenie/kategoria/14,administracja-biurowa HTTP/1.1" 200 33403 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.0; pl; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12 ( .NET CLR 3.5.30729; .$
103.219.23.28 - - [13/Oct/2017:03:53:37 +0200] "GET /ogloszenie/show/115324,doradca-klienta HTTP/1.1" 200 31672 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 OPR/22.0.$
79.110.28.43 - - [13/Oct/2017:03:53:35 +0200] "GET /ogloszenie/show/114847,osoba-od-dokumentacji-do-hardwareu-oraz-softwareu-niskopoziomowego HTTP/1.1" 200 31859 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
51.36.77.232 - - [13/Oct/2017:03:53:37 +0200] "GET /ogloszenie/region/4,podlaskie/strona/1 HTTP/1.1" 200 29814 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
47.29.16.32 - - [13/Oct/2017:03:53:37 +0200] "GET /ogloszenie/show/114823,kasjer--kasjerka HTTP/1.1" 200 28114 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
185.101.69.169 - - [13/Oct/2017:03:53:36 +0200] "GET /wiadomosc/60,moze-jednak-telepraca HTTP/1.1" 200 28067 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
210.123.58.97 - - [13/Oct/2017:03:53:38 +0200] "GET /praca/podkarpackie HTTP/1.1" 200 3503 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
46.161.58.49 - - [13/Oct/2017:03:53:38 +0200] "GET /ogloszenie/show/115415,account-manager HTTP/1.1" 200 28421 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
185.50.251.171 - - [13/Oct/2017:03:53:38 +0200] "GET /porady-i-wzory-cv?page=1 HTTP/1.1" 200 26939 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
5.62.155.104 - - [13/Oct/2017:03:53:36 +0200] "GET /wiadomosc/60,moze-jednak-telepraca HTTP/1.1" 200 28073 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
5.8.47.235 - - [13/Oct/2017:03:53:36 +0200] "GET /praca/podlaskie HTTP/1.1" 200 29813 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
82.178.103.7 - - [13/Oct/2017:03:53:39 +0200] "GET /porady-i-wzory-cv?page=2 HTTP/1.1" 200 30665 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
5.101.220.18 - - [13/Oct/2017:03:53:39 +0200] "GET /praca/podlaskie HTTP/1.1" 200 29809 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
185.106.104.231 - - [13/Oct/2017:03:53:39 +0200] "GET /praca/podkarpackie HTTP/1.1" 200 29712 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
58.136.194.192 - - [13/Oct/2017:03:53:39 +0200] "GET /ogloszenie/region/4,podlaskie/strona/4 HTTP/1.1" 200 137 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
172.221.211.53 - - [13/Oct/2017:03:53:50 +0200] "GET /wp-login.php HTTP/1.1" 500 3698 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1"
78.99.178.67 - - [13/Oct/2017:03:53:39 +0200] "GET /ogloszenie/kategoria/158,obsluga-klienta HTTP/1.1" 200 32943 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
5.188.219.70 - - [13/Oct/2017:03:53:38 +0200] "GET /praca/podkarpackie HTTP/1.1" 200 29709 "-" "Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144)"
38.95.191.244 - - [13/Oct/2017:03:53:39 +0200] "GET /ogloszenie/show/115165,pracownik-do-pomocy-przy-aranzacji-sklepu HTTP/1.1" 200 32003 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.1$
45.24.47.223 - - [13/Oct/2017:04:49:49 +0200] "GET / HTTP/1.1" 301 549 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1"
157.55.39.212 - - [13/Oct/2017:04:50:17 +0200] "GET /ogloszenie?SearchForm%5Bpracodawca%5D=5318 HTTP/1.1" 301 633 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingb
Click to expand...
 
What looks curious is the agent name in the logs: Apache-HttpClient/4.5.3-SNAPSHOT (Java/1.8.0_144). The same agent's name for IPs located in India, Great Britain, Saudi Arabia, Slovak Republic, etc.

I don't know your business and setup so I can't say whether or not they do any harm to your server. That might that there is no malicious activity on your server, and nobody attacks you. But you might have a badly written script which runs with a loop and occupies so much RAM. Check memory limits in PHP. Use server status, PHP-FPM status page to see which requests are repeatedly seen. Read the links which I already posted. Check PHP-FPM logs.

I suggested possible vectors for investigating, so you need to dive into it, and if you really want me to make a choice for you and fix the issue I need to connect to your server and do it myself.

The lines that you posted show only what they show, it's not sufficient for make to make any conclusion. Please do not post here full logs, I won' t check them all.

Here is another recent thread with a more or less similar issue https://forum.directadmin.com/showthread.php?t=55407
 
Thanks for this post! I'm going to make status page's and check it.
From php.ini I have
post_max_size = 224M
memory_limit = 258M
Give more?
 
You must log in or register to reply here.
Back
Top