Warning: The system load average is 103.21
- Thread starter mr-secret
- Start date
LawsHosting
Verified User
I get this too, but if you have a multi-core system, it's ok. It only lasts seconds.
It's ever since I changed to FPM it happens....
It's ever since I changed to FPM it happens....
LawsHosting
Verified User
I'm not too sure, I never had this issue when running php in mod_php & mod_ruid2, so it has to be something with FPM.....
Try asking in the PHP/MySQL sub-forum.
Try asking in the PHP/MySQL sub-forum.
zEitEr
Super Moderator
Hello,
For any issues with Apache+PHP you should start from analyzing of Apache server-status page. https://help.directadmin.com/item.php?id=91
Then check Apache server logs for a domain under question and search duplicated and repeated POST requests, your site might be under a brute-force or a spam-bots attack.
Check MySQL status with mysqladmin for long running queries and tables locks.
Use maldet and/or clamav for searching possible malware under public_html of users.
For any issues with Apache+PHP you should start from analyzing of Apache server-status page. https://help.directadmin.com/item.php?id=91
Then check Apache server logs for a domain under question and search duplicated and repeated POST requests, your site might be under a brute-force or a spam-bots attack.
Check MySQL status with mysqladmin for long running queries and tables locks.
Use maldet and/or clamav for searching possible malware under public_html of users.
i found problem but i cant resolve this
i check file
domain.com.log
many reqauest from more than100000 ips
i check file
domain.com.log
many reqauest from more than100000 ips
Code:
183.191.237.195 - - [18/May/2018:17:28:13 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl&direction=ltr HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
66.249.73.143 - - [18/May/2018:17:28:17 +0430] "GET /wp-content/uploads/2018/04/12122-1-300x300.jpg HTTP/1.0" 200 247 "-" "Googlebot-Image/1.0"
58.19.1.87 - - [18/May/2018:17:28:13 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl&direction=rtl HTTP/1.0" 404 6051 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
171.109.46.42 - - [18/May/2018:17:28:13 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr&direction=ltr HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
222.243.211.113 - - [18/May/2018:17:28:14 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1269 HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
39.71.131.216 - - [18/May/2018:17:28:15 +0430] "GET /product/??????-???-???-????-????/?add-to-cart=1249 HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
110.230.197.122 - - [18/May/2018:17:28:15 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
180.143.244.42 - - [18/May/2018:17:28:15 +0430] "GET /product/??????-???-???-????-????/ HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
58.19.1.87 - - [18/May/2018:17:28:15 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1269 HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
117.158.179.177 - - [18/May/2018:17:28:16 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249 HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
113.237.244.18 - - [18/May/2018:17:28:16 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1389 HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
113.108.126.6 - - [18/May/2018:17:28:17 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr&direction=ltr&direction=rtl HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
218.58.137.186 - - [18/May/2018:17:28:17 +0430] "GET /product/??????-???-???-????-????/ HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
116.21.62.202 - - [18/May/2018:17:28:17 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
114.243.243.64 - - [18/May/2018:17:28:18 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1269 HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
219.149.193.198 - - [18/May/2018:17:28:18 +0430] "GET /product/??????-???-???-????-????/?add-to-cart=1284 HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
27.204.50.38 - - [18/May/2018:17:28:18 +0430] "GET /product/??????-???-???-????-????/?add-to-cart=1284 HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
171.109.46.42 - - [18/May/2018:17:28:19 +0430] "GET /product/??????-???-???-????-????/ HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
111.172.232.102 - - [18/May/2018:17:28:19 +0430] "GET /product/??????-???-???-????-????/?add-to-cart=1249 HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
118.239.9.175 - - [18/May/2018:17:28:19 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432 HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
119.144.124.21 - - [18/May/2018:17:28:20 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249 HTTP/1.0" 404 6056 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
59.27.232.32 - - [18/May/2018:17:28:20 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432 HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
36.40.14.38 - - [18/May/2018:17:28:21 +0430] "GET /product/??????-???-???-????-????/?add-to-cart=1284 HTTP/1.0" 404 6056 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
42.63.236.214 - - [18/May/2018:17:28:21 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
182.33.162.209 - - [18/May/2018:17:28:21 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1269 HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
110.230.197.122 - - [18/May/2018:17:28:23 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1389&direction=rtl HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
112.102.137.228 - - [18/May/2018:17:28:24 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl&direction=ltr HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
117.13.144.85 - - [18/May/2018:17:28:24 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
119.144.124.21 - - [18/May/2018:17:28:25 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr HTTP/1.0" 404 6052 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
42.63.236.214 - - [18/May/2018:17:28:25 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1389&direction=rtl HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
42.63.236.214 - - [18/May/2018:17:28:26 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl&direction=ltr HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
123.163.152.12 - - [18/May/2018:17:28:28 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249 HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
123.152.246.111 - - [18/May/2018:17:28:28 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr&direction=ltr&direction=rtl HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
117.158.179.177 - - [18/May/2018:17:28:28 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
171.127.249.53 - - [18/May/2018:17:28:29 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windo
ReN
Verified User
- Joined
- Jul 2, 2005
- Messages
- 185
Greetings , from this little bit of info you gave us , it maybe a dos attack on that domain, which web server are using, if its apache , look into mod_evasive , this might limit it enough for you
as and extra layer , look into fail2ban , this will help with individual ips or ranges.
Let us know how you go.
zEitEr
Super Moderator
A referer is empty, you probably could block all connections with add_to_wishlist= and empty referer.
0 (!). Consider adding more ram to the server. You have 4G and what we see on the last attachment is that under normal load you use 3/4 of it. Probably when you are under DDoS attack you get out of ram and you swap. I am pretty sure that that's the explanation for the high load - when your server swaps, it's making everything via disk operations and your hard drives are not fast enough to keep up the requests (happens even on SSD, and happens always on HDD)..
1. Tune up your firewall. Don't allow too many connections from one IP.
2. Lower all your timeouts. This includes the apache general timeout, keep alive time in apache, mysql idle timeout (just in case there are badly written scripts which are attacked)
3. Tune up your network card settings via sysctl.conf.
4. Think of implementing caching strategies for larger websites which use lots of resources
You can't stop DDoS attacks but you can at least make the server more durable.
1. Tune up your firewall. Don't allow too many connections from one IP.
2. Lower all your timeouts. This includes the apache general timeout, keep alive time in apache, mysql idle timeout (just in case there are badly written scripts which are attacked)
3. Tune up your network card settings via sysctl.conf.
4. Think of implementing caching strategies for larger websites which use lots of resources
You can't stop DDoS attacks but you can at least make the server more durable.
Last edited:
hello , i have same problem and all my websites are down now when i look on TOP i see there is no load and ram is enough
top - 10:59:46 up 1:22, 1 user, load average: 0.10, 0.12, 0.09
Tasks: 333 total, 1 running, 332 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.1 us, 0.1 sy, 0.0 ni, 99.8 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 5774.5 total, 3649.8 free, 937.1 used, 1187.6 buff/cache
MiB Swap: 2112.0 total, 2112.0 free, 0.0 used. 4560.6 avail Mem
top - 10:59:46 up 1:22, 1 user, load average: 0.10, 0.12, 0.09
Tasks: 333 total, 1 running, 332 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.1 us, 0.1 sy, 0.0 ni, 99.8 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 5774.5 total, 3649.8 free, 937.1 used, 1187.6 buff/cache
MiB Swap: 2112.0 total, 2112.0 free, 0.0 used. 4560.6 avail Mem