Warning: The system load average is 12.12

taker18 · May 6, 2024

Is this warning about one site or the entire server?

What should I do about it? I never got to add swap memory.
any advice, please would appreciate it
sounds like of the top memory usage is by one of the websites
but I increased PHP memory limited to 1024M.
THank you

Warning: The system load average is 12.12

5/01/2024, 12:45 PM
This is an automated message notifying you that the 5 minute load average on your system is 12.12.
This has exceeded the 10 threshold.

One Minute - 27.47
Five Minutes - 12.12
Fifteen Minutes - 5.48

top - 17:45:40 up 28 days, 12:02, 0 users, load average: 27.47, 12.12, 5.48
Tasks: 232 total, 5 running, 227 sleeping, 0 stopped, 0 zombie
%Cpu(s): 57.5 us, 42.5 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 15355.2 total, 3521.6 free, 4601.6 used, 7875.9 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 10753.7 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2087898 admin 20 0 287916 210352 17256 R 89.5 1.3 0:09.40 lsphp:tute.org/public_html/wp-admin/admin-ajax.php
2087907 admin 20 0 314348 163024 15280 R 89.5 1.0 0:08.11 lsphp:tute.org/public_html/wp-admin/admin-ajax.php
2087968 admin 20 0 302156 218300 11000 R 84.2 1.4 0:06.58 lsphp:ainstitute.org/public_html/wp-admin/post.php
2087959 admin 20 0 265292 183188 12756 R 73.7 1.2 0:05.72 lsphp:tute.org/public_html/wp-admin/admin-ajax.php
761 mysql 20 0 2648436 697848 18008 S 5.3 4.4 169:28.77 /usr/local/mysql/bin/mysqld --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock
2088047 root 20 0 7636 3652 3016 R 5.3 0.0 0:00.02 /usr/bin/top -c -b -n 1
1 root 20 0 170728 12436 7600 S 0.0 0.1 3:42.37 /usr/lib/systemd/systemd --switched-root --system --deserialize 31
2 root 20 0 0 0 0 S 0.0 0.0 0:00.77 [kthreadd]
3 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 [rcu_gp]
4 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 [rcu_par_gp]
5 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 [slub_flushwq]
6 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 [netns]
8 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 [kworker/0:0H-events_highpri]
10 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 [mm_percpu_wq]
12 root 20 0 0 0 0 I 0.0 0.0 0:00.00 [rcu_tasks_kthre]
13 root 20 0 0 0 0 I 0.0 0.0 0:00.00 [rcu_tasks_rude_]
14 root 20 0 0 0 0 I 0.0 0.0 0:00.00 [rcu_tasks_trace]
15 root 20 0 0 0 0 S 0.0 0.0 0:38.59 [ksoftirqd/0]
16 root 20 0 0 0 0 S 0.0 0.0 20:18.32 [pr/ttyS0]
17 root 20 0 0 0 0 S 0.0 0.0 32:47.00 [pr/tty1]
18 root 20 0 0 0 0 I 0.0 0.0 25:40.81 [rcu_preempt]
19 root rt 0 0 0 0 S 0.0 0.0 0:13.23 [migration/0]
20 root -51 0 0 0 0 S 0.0 0.0 0:00.00 [idle_inject/0]

Top Memory Usage:
mysql 761 0.4 4.4 2648436 697848 ? Ssl Apr03 169:28 /usr/local/mysql/bin/mysqld --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock
admin 2087968 15.7 1.4 304560 226152 ? Rs 17:44 0:06 lsphp:ainstitute.org/public_html/wp-admin/post.php
admin 2087898 13.5 1.3 289964 212400 ? Rs 17:44 0:09 lsphp:tute.org/public_html/wp-admin/admin-ajax.php
admin 2087959 12.3 1.1 269388 187284 ? Rs 17:44 0:05 lsphp:tute.org/public_html/wp-admin/admin-ajax.php
admin 2084565 6.6 1.1 319180 174968 ? Ss 16:16 5:53 lsphp
admin 2087916 10.4 1.0 309628 163696 ? Ss 17:44 0:06 lsphp
admin 2087910 9.2 1.0 308360 162604 ? Ss 17:44 0:06 lsphp
admin 2087907 12.0 1.0 319468 166128 ? Rs 17:44 0:08 lsphp:tute.org/public_html/wp-admin/admin-ajax.php
admin 2087904 9.7 1.0 235228 159780 ? Ss 17:44 0:06 lsphp
admin 2087897 9.5 1.0 234796 159164 ? Ss 17:44 0:06 lsphp

Virtual Memory Info:
--procs-- -----------------------memory---------------------- ---swap-- -----io---- -system-- --------cpu-------- -----timestamp-----
r b swpd free buff cache si so bi bo in cs us sy id wa st UTC
4 0 0 3588984 240412 7824500 0 0 11 282 0 4 3 2 95 0 0 2024-05-01 17:45:40
1 0 0 4149652 240412 7824764 0 0 0 156 4391 2804 39 32 30 0 0 2024-05-01 17:45:41
2 0 0 4192240 240412 7824768 0 0 0 40 2865 1389 24 17 59 0 0 2024-05-01 17:45:42

zEitEr · May 7, 2024

Hello,

This is a warning about the entire server. The server load went too high and directadmin warned you about it. The causes of it might be one or several of the following:

- brute-force attacks
- httpd-flood
- spam bots abusing contact forms
- search engines and fake search bots
- vulnerability scanners
- etc.

Actions that you might apply will depend on a source of the issue:

- enable brute-force monitor in directadmin
- block too aggressive bots
- limit resources on per user bases
- etc

You can do it by yourself, the forums contain guides for some of the cases. Some can be found here: https://docs.directadmin.com/ and https://help.poralix.com/

Also you might hire somebody to take care of your server.

Active8 · May 7, 2024

+ hard disk failure is also possible

taker18 · May 7, 2024

zEitEr said:
Hello,

This is a warning about the entire server. The server load went too high and directadmin warned you about it. The causes of it might be one or several of the following:

sounds like I need to investigation more about
Thank you all for the advice

zEitEr · May 7, 2024

taker18 said:
57.5 us, 42.5 sy, 0.0 ni, 0.0 id, 0.0 wa

IO/wait is zero here, but user-mode and system CPU are a bit high. The following processes are using CPU too much:

Code:

2087898 admin 20 0 287916 210352 17256 R 89.5 1.3 0:09.40 lsphp:tute.org/public_html/wp-admin/admin-ajax.php
2087907 admin 20 0 314348 163024 15280 R 89.5 1.0 0:08.11 lsphp:tute.org/public_html/wp-admin/admin-ajax.php
2087968 admin 20 0 302156 218300 11000 R 84.2 1.4 0:06.58 lsphp:ainstitute.org/public_html/wp-admin/post.php
2087959 admin 20 0 265292 183188 12756 R 73.7 1.2 0:05.72 lsphp:tute.org/public_html/wp-admin/admin-ajax.php

It seems an admin is working in WP dashboard, posting a new publication or editing an existing one. The mentioned processes can be either the culprit of the high load or be a result of high CPU usage. A snapshot of running processes in a DirectAdmin report might miss a real culprit. That should be investigated using system logs.

taker18 · May 7, 2024

zEitEr said:
IO/wait is zero here, but user-mode and system CPU are a bit high. The following processes are using CPU too much:

Code:

2087898 admin 20 0 287916 210352 17256 R 89.5 1.3 0:09.40 lsphp:tute.org/public_html/wp-admin/admin-ajax.php 2087907 admin 20 0 314348 163024 15280 R 89.5 1.0 0:08.11 lsphp:tute.org/public_html/wp-admin/admin-ajax.php 2087968 admin 20 0 302156 218300 11000 R 84.2 1.4 0:06.58 lsphp:ainstitute.org/public_html/wp-admin/post.php 2087959 admin 20 0 265292 183188 12756 R 73.7 1.2 0:05.72 lsphp:tute.org/public_html/wp-admin/admin-ajax.php

It seems an admin is working in WP dashboard, posting a new publication or editing an existing one. The mentioned processes can be either the culprit of the high load or be a result of high CPU usage. A snapshot of running processes in a DirectAdmin report might miss a real culprit. That should be investigated using system logs.

I think you got this very right @zEitEr. I suspected that this site is causing this heavy load, which has heavy course Materials.
But the question how can I measure the activity of this single website? It sounds like using some CPU in some time more than others.
the Question is
How can I follow the activity of this single website compared to others? In the direct admin Panel? Or on the website itself?

zEitEr · May 7, 2024

taker18 said:
How can I follow the activity of this single website compared to others? In the direct admin Panel? Or on the website itself?

Usually CPU load goes alongside with long running processes. In the case they are PHP processes. You could start with PHP slow logs if you used PHP-FPM. But as far as I can see you use LSPHP. Is it LiteSpeed or OpenLiteSpeed server by the way?

For LiteSpeed you might check this: https://www.litespeedtech.com/support/wiki/doku.php/litespeed_wiki:php:slow_log_lsapi and this https://docs.litespeedtech.com/lsws/extapp/php/troubleshooting/

Also you might use strace -p PID to see in a real time, what a process does. You will need to have certain expertise to understand the output.

As another way to go is to try and disable WP addons one by one, and see how the things go.

taker18 · May 9, 2024

zEitEr said:
Usually CPU load goes alongside with long running processes. In the case they are PHP processes. You could start with PHP slow logs if you used PHP-FPM. But as far as I can see you use LSPHP. Is it LiteSpeed or OpenLiteSpeed server by the way?
As another way to go is to try and disable WP addons one by one, and see how the things go.

Here is the Update:
Thank you so much @zEitEr for confirming my suspicions that this one particular website is taking 60% to 90% of the CPU load, not as much as the RAM, really. which is strange for me, that properly because of the amount of dynamic content that I have on this site . It's pretty much a website issue, not a server. I checked
Simply I used the Top Command and suspended this website from DA, and all of a sudden, the CPU went down to less than 10%. now I know that I will work around one single website only. which is a lot easier to deal with as of now now.
issue solved
Great support Direct Admin Community

Warning: The system load average is 12.12

taker18

Verified User

Is this warning about one site or the entire server?