Results 1 to 9 of 9

Thread: Warning: The system load average is 103.21

  1. #1
    Join Date
    Jan 2015
    Posts
    31

    Exclamation Warning: The system load average is 103.21

    Hello

    on user on my hosting run many proccess and directadmin send this message Warning: The system load average is 103.21
    i check user files but cant find any unnormally files
    proccess picture atached.
    Attached Images Attached Images

  2. #2
    Join Date
    Sep 2008
    Location
    London UK
    Posts
    1,654
    I get this too, but if you have a multi-core system, it's ok. It only lasts seconds.

    It's ever since I changed to FPM it happens....

  3. #3
    Join Date
    Jan 2015
    Posts
    31
    this get hang server after 20-30 mintes
    when syspend user everything is ok
    how can resolve this problem?

  4. #4
    Join Date
    Sep 2008
    Location
    London UK
    Posts
    1,654
    I'm not too sure, I never had this issue when running php in mod_php & mod_ruid2, so it has to be something with FPM.....

    Try asking in the PHP/MySQL sub-forum.

  5. #5
    Join Date
    Apr 2005
    Location
    GMT +7.00
    Posts
    12,471
    Hello,

    For any issues with Apache+PHP you should start from analyzing of Apache server-status page. https://help.directadmin.com/item.php?id=91

    Then check Apache server logs for a domain under question and search duplicated and repeated POST requests, your site might be under a brute-force or a spam-bots attack.

    Check MySQL status with mysqladmin for long running queries and tables locks.

    Use maldet and/or clamav for searching possible malware under public_html of users.

  6. #6
    Join Date
    Jan 2015
    Posts
    31
    i found problem but i cant resolve this
    i check file
    domain.com.log
    many reqauest from more than100000 ips

    Code:
    183.191.237.195 - - [18/May/2018:17:28:13 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl&direction=ltr HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    66.249.73.143 - - [18/May/2018:17:28:17 +0430] "GET /wp-content/uploads/2018/04/12122-1-300x300.jpg HTTP/1.0" 200 247 "-" "Googlebot-Image/1.0"
    58.19.1.87 - - [18/May/2018:17:28:13 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl&direction=rtl HTTP/1.0" 404 6051 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    171.109.46.42 - - [18/May/2018:17:28:13 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr&direction=ltr HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    222.243.211.113 - - [18/May/2018:17:28:14 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1269 HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    39.71.131.216 - - [18/May/2018:17:28:15 +0430] "GET /product/??????-???-???-????-????/?add-to-cart=1249 HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    110.230.197.122 - - [18/May/2018:17:28:15 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    180.143.244.42 - - [18/May/2018:17:28:15 +0430] "GET /product/??????-???-???-????-????/ HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    58.19.1.87 - - [18/May/2018:17:28:15 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1269 HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    117.158.179.177 - - [18/May/2018:17:28:16 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249 HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    113.237.244.18 - - [18/May/2018:17:28:16 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1389 HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    113.108.126.6 - - [18/May/2018:17:28:17 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr&direction=ltr&direction=rtl HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    218.58.137.186 - - [18/May/2018:17:28:17 +0430] "GET /product/??????-???-???-????-????/ HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    116.21.62.202 - - [18/May/2018:17:28:17 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    114.243.243.64 - - [18/May/2018:17:28:18 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1269 HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    219.149.193.198 - - [18/May/2018:17:28:18 +0430] "GET /product/??????-???-???-????-????/?add-to-cart=1284 HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    27.204.50.38 - - [18/May/2018:17:28:18 +0430] "GET /product/??????-???-???-????-????/?add-to-cart=1284 HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    171.109.46.42 - - [18/May/2018:17:28:19 +0430] "GET /product/??????-???-???-????-????/ HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    111.172.232.102 - - [18/May/2018:17:28:19 +0430] "GET /product/??????-???-???-????-????/?add-to-cart=1249 HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    118.239.9.175 - - [18/May/2018:17:28:19 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432 HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    119.144.124.21 - - [18/May/2018:17:28:20 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249 HTTP/1.0" 404 6056 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    59.27.232.32 - - [18/May/2018:17:28:20 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432 HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    36.40.14.38 - - [18/May/2018:17:28:21 +0430] "GET /product/??????-???-???-????-????/?add-to-cart=1284 HTTP/1.0" 404 6056 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    42.63.236.214 - - [18/May/2018:17:28:21 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    182.33.162.209 - - [18/May/2018:17:28:21 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1269 HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    110.230.197.122 - - [18/May/2018:17:28:23 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1389&direction=rtl HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    112.102.137.228 - - [18/May/2018:17:28:24 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl&direction=ltr HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    117.13.144.85 - - [18/May/2018:17:28:24 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    119.144.124.21 - - [18/May/2018:17:28:25 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr HTTP/1.0" 404 6052 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    42.63.236.214 - - [18/May/2018:17:28:25 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1389&direction=rtl HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    42.63.236.214 - - [18/May/2018:17:28:26 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl&direction=ltr HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    123.163.152.12 - - [18/May/2018:17:28:28 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249 HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    123.152.246.111 - - [18/May/2018:17:28:28 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr&direction=ltr&direction=rtl HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    117.158.179.177 - - [18/May/2018:17:28:28 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    171.127.249.53 - - [18/May/2018:17:28:29 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windo

  7. #7
    Join Date
    Jul 2005
    Posts
    168
    Quote Originally Posted by mr-secret View Post
    i found problem but i cant resolve this
    i check file
    domain.com.log
    many reqauest from more than100000 ips

    Code:
    183.191.237.195 - - [18/May/2018:17:28:13 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl&direction=ltr HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    66.249.73.143 - - [18/May/2018:17:28:17 +0430] "GET /wp-content/uploads/2018/04/12122-1-300x300.jpg HTTP/1.0" 200 247 "-" "Googlebot-Image/1.0"
    58.19.1.87 - - [18/May/2018:17:28:13 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl&direction=rtl HTTP/1.0" 404 6051 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    171.109.46.42 - - [18/May/2018:17:28:13 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr&direction=ltr HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    222.243.211.113 - - [18/May/2018:17:28:14 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1269 HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    39.71.131.216 - - [18/May/2018:17:28:15 +0430] "GET /product/??????-???-???-????-????/?add-to-cart=1249 HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    110.230.197.122 - - [18/May/2018:17:28:15 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    180.143.244.42 - - [18/May/2018:17:28:15 +0430] "GET /product/??????-???-???-????-????/ HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    58.19.1.87 - - [18/May/2018:17:28:15 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1269 HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    117.158.179.177 - - [18/May/2018:17:28:16 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249 HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    113.237.244.18 - - [18/May/2018:17:28:16 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1389 HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    113.108.126.6 - - [18/May/2018:17:28:17 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr&direction=ltr&direction=rtl HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    218.58.137.186 - - [18/May/2018:17:28:17 +0430] "GET /product/??????-???-???-????-????/ HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    116.21.62.202 - - [18/May/2018:17:28:17 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    114.243.243.64 - - [18/May/2018:17:28:18 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1269 HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    219.149.193.198 - - [18/May/2018:17:28:18 +0430] "GET /product/??????-???-???-????-????/?add-to-cart=1284 HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    27.204.50.38 - - [18/May/2018:17:28:18 +0430] "GET /product/??????-???-???-????-????/?add-to-cart=1284 HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    171.109.46.42 - - [18/May/2018:17:28:19 +0430] "GET /product/??????-???-???-????-????/ HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    111.172.232.102 - - [18/May/2018:17:28:19 +0430] "GET /product/??????-???-???-????-????/?add-to-cart=1249 HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    118.239.9.175 - - [18/May/2018:17:28:19 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432 HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    119.144.124.21 - - [18/May/2018:17:28:20 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249 HTTP/1.0" 404 6056 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    59.27.232.32 - - [18/May/2018:17:28:20 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432 HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    36.40.14.38 - - [18/May/2018:17:28:21 +0430] "GET /product/??????-???-???-????-????/?add-to-cart=1284 HTTP/1.0" 404 6056 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    42.63.236.214 - - [18/May/2018:17:28:21 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    182.33.162.209 - - [18/May/2018:17:28:21 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1269 HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    110.230.197.122 - - [18/May/2018:17:28:23 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1389&direction=rtl HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    112.102.137.228 - - [18/May/2018:17:28:24 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl&direction=ltr HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    117.13.144.85 - - [18/May/2018:17:28:24 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    119.144.124.21 - - [18/May/2018:17:28:25 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr HTTP/1.0" 404 6052 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    42.63.236.214 - - [18/May/2018:17:28:25 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1389&direction=rtl HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    42.63.236.214 - - [18/May/2018:17:28:26 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl&direction=ltr HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    123.163.152.12 - - [18/May/2018:17:28:28 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249 HTTP/1.0" 404 6055 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    123.152.246.111 - - [18/May/2018:17:28:28 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr&direction=ltr&direction=rtl HTTP/1.0" 404 6053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    117.158.179.177 - - [18/May/2018:17:28:28 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1249&direction=rtl&direction=ltr HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
    171.127.249.53 - - [18/May/2018:17:28:29 +0430] "GET /product/??????-???-???-????-????/?add_to_wishlist=1432&direction=rtl HTTP/1.0" 404 6054 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windo

    Greetings , from this little bit of info you gave us , it maybe a dos attack on that domain, which web server are using, if its apache , look into mod_evasive , this might limit it enough for you
    as and extra layer , look into fail2ban , this will help with individual ips or ranges.

    Let us know how you go.

  8. #8
    Join Date
    Apr 2005
    Location
    GMT +7.00
    Posts
    12,471
    A referer is empty, you probably could block all connections with add_to_wishlist= and empty referer.

  9. #9
    Join Date
    May 2008
    Posts
    755
    0 (!). Consider adding more ram to the server. You have 4G and what we see on the last attachment is that under normal load you use 3/4 of it. Probably when you are under DDoS attack you get out of ram and you swap. I am pretty sure that that's the explanation for the high load - when your server swaps, it's making everything via disk operations and your hard drives are not fast enough to keep up the requests (happens even on SSD, and happens always on HDD)..

    1. Tune up your firewall. Don't allow too many connections from one IP.

    2. Lower all your timeouts. This includes the apache general timeout, keep alive time in apache, mysql idle timeout (just in case there are badly written scripts which are attacked)

    3. Tune up your network card settings via sysctl.conf.

    4. Think of implementing caching strategies for larger websites which use lots of resources

    You can't stop DDoS attacks but you can at least make the server more durable.
    Last edited by wattie; 05-23-2018 at 12:50 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •