system average load exceeded the threshold

A

ajax20

Verified User
Joined
Jul 16, 2014
Messages
142
Hi

Today I noticed I have around 65 alerts in DA saying that the system average load exceeded the threshold on my server yesterday. Upon checking, I came to lots of lines in my log file which I am trying to make sense of. I am worried if my server has already been contaminated but this might as well be just some malfunctioning somewhere related to lfd or clamav.

The alerts I get in my mails range from clamav

Exceeded: 1614931 > 1800 (seconds)

to

lfd checking SYSLOG Check

Error: Failed to detect code [EsURjDjwAT2x4XA] in SYSLOG_LOG [/var/log/messages]

I also get the following lines in my log file high are related to my dns. I did not sea these errors before and I do not have any ideas why I get the network unreachable errors. Why does my server try to resolve to these addresses at all?

Code: 
Oct 23 11:39:03 server named[1585]: error (network unreachable) resolving './DNSKEY/IN': 2001:503:ba3e::2:30#53
Oct 23 11:39:03 server named[1585]: error (network unreachable) resolving './NS/IN': 2001:503:ba3e::2:30#53
Oct 23 11:39:03 server named[1585]: error (network unreachable) resolving 'dlv.isc.org/DNSKEY/IN': 2001:500:48::1#53
Oct 23 11:39:03 server named[1585]: error (network unreachable) resolving 'dlv.isc.org/DNSKEY/IN': 2001:4f8:0:2::19#53
Oct 23 11:39:03 server named[1585]: error (network unreachable) resolving 'ns.isc.afilias-nst.info/A/IN': 2001:500:2f::f#53
Oct 23 11:39:03 server named[1585]: error (network unreachable) resolving 'ns.isc.afilias-nst.info/AAAA/IN': 2001:500:2f::f#53
Oct 23 11:39:03 server named[1585]: error (network unreachable) resolving 'ns.isc.afilias-nst.info/A/IN': 2001:500:1::803f:235#53
Oct 23 11:39:03 server named[1585]: error (network unreachable) resolving 'ns.isc.afilias-nst.info/AAAA/IN': 2001:500:1::803f:235#53
Oct 23 11:39:03 server named[1585]: error (network unreachable) resolving 'ns.isc.afilias-nst.info/A/IN': 2001:503:c27::2:30#53
Oct 23 11:39:03 server named[1585]: error (network unreachable) resolving 'ns.isc.afilias-nst.info/AAAA/IN': 2001:503:c27::2:30#53
Oct 23 11:39:03 server named[1585]: error (network unreachable) resolving 'ns.isc.afilias-nst.info/A/IN': 2001:500:1a::1#53
Oct 23 11:39:03 server named[1585]: error (network unreachable) resolving 'dlv.isc.org/DNSKEY/IN': 2001:4f8:0:2::20#53
Oct 23 11:39:03 server named[1585]: error (network unreachable) resolving 'dlv.isc.org/DNSKEY/IN': 2001:500:60::29#53
Oct 23 11:39:03 server named[1585]: error (network unreachable) resolving 'ns1.isc.ultradns.net/A/IN': 2001:7fd::1#53
Oct 23 11:39:03 server named[1585]: error (network unreachable) resolving 'ns1.isc.ultradns.net/AAAA/IN': 2001:7fd::1#53
Oct 23 11:39:04 server named[1585]: error (network unreachable) resolving 'ns2.isc.ultradns.net/A/IN': 2610:a1:1014::e8#53
Oct 23 11:39:04 server named[1585]: error (network unreachable) resolving 'pdns196.ultradns.org/A/IN': 2001:500:e::1#53
Oct 23 11:39:04 server named[1585]: error (network unreachable) resolving 'pdns196.ultradns.org/AAAA/IN': 2001:500:e::1#53
Oct 23 11:39:04 server named[1585]: error (network unreachable) resolving 'pdns196.ultradns.org/A/IN': 2001:500:40::1#53
Oct 23 11:39:04 server named[1585]: error (network unreachable) resolving 'pdns196.ultradns.org/AAAA/IN': 2001:500:40::1#53
Oct 23 11:39:04 server named[1585]: error (network unreachable) resolving 'pdns196.ultradns.org/AAAA/IN': 2001:502:4612::e8#53
Oct 23 11:39:04 server named[1585]: error (network unreachable) resolving 'pdns196.ultradns.info/AAAA/IN': 2610:a1:1016::e8#53
Oct 23 11:39:04 server named[1585]: error (network unreachable) resolving 'pdns196.ultradns.info/A/IN': 2610:a1:1016::e8#53
Oct 23 11:39:04 server named[1585]: error (network unreachable) resolving 'pdns196.ultradns.co.uk/AAAA/IN': 2610:a1:1017::e8#53
Oct 23 11:39:04 server named[1585]: error (network unreachable) resolving 'pdns196.ultradns.biz/A/IN': 2610:a1:1015::e8#53
Oct 23 11:39:04 server named[1585]: error (network unreachable) resolving 'pdns196.ultradns.com/AAAA/IN': 2001:502:f3ff::e8#53
Oct 23 11:39:04 server named[1585]: client 93.113.174.225#46368: query (cache) 'adobe.com/A/IN' denied
Oct 23 11:39:04 server named[1585]: client 93.113.174.225#23736: query (cache) 'adobe.com/A/IN' denied
Oct 23 11:39:04 server lfd[1196]: SYSLOG check [Lga6AZUNsgZGaVQX]

Please help!

Thanks
 
I also have no idea who client 93.113.174.225 is. That is not my IP address.
 
My log file also contains the following lines which I do not make sense of as I did not have any updates yesterday.

Code: 
Oct 22 08:04:22 server kernel: lfserverd invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
Oct 22 08:04:22 server kernel: lfd cpuset=/ mems_allowed=0
Oct 22 08:04:22 server kernel: Pid: 12683, comm: lfd Not tainted 2.6.32-431.29.2.el6.x86_64 #1
Oct 22 08:04:22 server kernel: Call Trace:
Oct 22 08:04:22 server kernel: [<ffffffff810d0361>] ? cpuset_print_task_mems_allowed+0x91/0xb0
Oct 22 08:04:22 server kernel: [<ffffffff81122730>] ? dump_header+0x90/0x1b0
Oct 22 08:04:22 server kernel: [<ffffffff812282fc>] ? security_real_capable_noaudit+0x3c/0x70
Oct 22 08:04:22 server kernel: [<ffffffff81122bb2>] ? oom_kill_process+0x82/0x2a0
Oct 22 08:04:22 server kernel: [<ffffffff81122af1>] ? select_bad_process+0xe1/0x120
Oct 22 08:04:22 server kernel: [<ffffffff81122ff0>] ? out_of_memory+0x220/0x3c0
Oct 22 08:04:22 server kernel: [<ffffffff8112f90f>] ? __alloc_pages_nodemask+0x89f/0x8d0
Oct 22 08:04:22 server kernel: [<ffffffff8116799a>] ? alloc_pages_vma+0x9a/0x150
Oct 22 08:04:22 server kernel: [<ffffffff8114958d>] ? do_wp_page+0xfd/0x920
Oct 22 08:04:22 server kernel: [<ffffffff8115c393>] ? swap_info_get+0x63/0xe0
Oct 22 08:04:22 server kernel: [<ffffffff8114a5ad>] ? handle_pte_fault+0x2cd/0xb00
Oct 22 08:04:22 server kernel: [<ffffffff8114b00a>] ? handle_mm_fault+0x22a/0x300
Oct 22 08:04:22 server kernel: [<ffffffff8104a8d8>] ? __do_page_fault+0x138/0x480
Oct 22 08:04:22 server kernel: [<ffffffff8100988e>] ? __switch_to+0x26e/0x320
Oct 22 08:04:22 server kernel: [<ffffffff81528e4e>] ? thread_return+0x4e/0x770
Oct 22 08:04:22 server kernel: [<ffffffff8100bb8e>] ? apic_timer_interrupt+0xe/0x20
Oct 22 08:04:22 server kernel: [<ffffffff8152e99e>] ? do_page_fault+0x3e/0xa0
Oct 22 08:04:22 server kernel: [<ffffffff8152bd55>] ? page_fault+0x25/0x30
Oct 22 08:04:22 server kernel: Mem-Info:
Oct 22 08:04:22 server kernel: Node 0 DMA per-cpu:
Oct 22 08:04:22 server kernel: CPU    0: hi:    0, btch:   1 usd:   0
Oct 22 08:04:22 server kernel: Node 0 DMA32 per-cpu:
Oct 22 08:04:22 server kernel: CPU    0: hi:  186, btch:  31 usd:  99
Oct 22 08:04:22 server kernel: active_anon:105167 inactive_anon:107051 isolated_anon:2816
Oct 22 08:04:22 server kernel: active_file:213 inactive_file:285 isolated_file:0
Oct 22 08:04:22 server kernel: unevictable:1000 dirty:0 writeback:237 unstable:0
Oct 22 08:04:22 server kernel: free:12249 slab_reclaimable:2674 slab_unreclaimable:8197
Oct 22 08:04:22 server kernel: mapped:849 shmem:8 pagetables:9534 bounce:0
Oct 22 08:04:22 server kernel: Node 0 DMA free:4644kB min:668kB low:832kB high:1000kB active_anon:976kB inactive_anon:9248kB active_file:0kB inactive_file:200kB unevictable:0kB iso$
Oct 22 08:04:22 server kernel: lowmem_reserve[]: 0 994 994 994
Oct 22 08:04:22 server kernel: Node 0 DMA32 free:44352kB min:44384kB low:55480kB high:66576kB active_anon:419692kB inactive_anon:418956kB active_file:852kB inactive_file:940kB unev$
Oct 22 08:04:22 server kernel: lowmem_reserve[]: 0 0 0 0
Oct 22 08:04:22 server kernel: Node 0 DMA: 15*4kB 13*8kB 8*16kB 6*32kB 9*64kB 6*128kB 3*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 4644kB
Oct 22 08:04:22 server kernel: Node 0 DMA32: 224*4kB 812*8kB 704*16kB 271*32kB 110*64kB 48*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 44352kB
Oct 22 08:04:22 server kernel: 27024 total pagecache pages
Oct 22 08:04:22 server kernel: 25794 pages in swap cache
Oct 22 08:04:22 server kernel: Swap cache stats: add 7572746, delete 7546952, find 374425170/374964258
Oct 22 08:04:22 server kernel: Free swap  = 0kB
Oct 22 08:04:22 server kernel: Total swap = 2588668kB
Oct 22 08:04:22 server kernel: 262141 pages RAM
Oct 22 08:04:22 server kernel: 7042 pages reserved
Oct 22 08:04:27 server kernel: 58117 pages shared
Oct 22 08:04:27 server kernel: 233290 pages non-shared
Oct 22 08:04:27 server kernel: [ pid ]   uid  tgid total_vm      rss cpu oom_adj oom_score_adj name
Oct 22 08:04:27 server kernel: [  367]     0   367     2732	65   0     -17         -1000 udevd
Oct 22 08:04:27 server kernel: [  970]     0   970    23300      143   0     -17         -1000 auditd
Oct 22 08:04:27 server kernel: [ 1011]     0  1011    62368      253   0       0             0 rsyslogd
Oct 22 08:04:27 server kernel: [ 1233]     0  1233     5665      136   0       0             0 dovecot
Oct 22 08:04:27 server kernel: [ 1246]   494  1246    11319      153   0       0             0 imap-login
Oct 22 08:04:27 server kernel: [ 1247]   494  1247     4090      136   0       0             0 anvil
Oct 22 08:04:27 server kernel: [ 1248]     0  1248     4123      142   0       0             0 log
Oct 22 08:04:27 server kernel: [ 1260]   494  1260    11319      153   0       0             0 imap-login
Oct 22 08:04:27 server kernel: [ 1261]   494  1261    11319      153   0       0             0 imap-login
Oct 22 08:04:27 server kernel: [ 1262]   494  1262    11319      153   0       0             0 imap-login
Oct 22 08:04:27 server kernel: [ 1263]   494  1263    11319      153   0       0             0 imap-login
Oct 22 08:04:27 server kernel: [ 1264]   494  1264    11319      153   0       0             0 imap-login
Oct 22 08:04:27 server kernel: [ 1265]   494  1265    11319      153   0       0             0 imap-login
Oct 22 08:04:27 server kernel: [ 1266]   494  1266    11319      153   0       0             0 imap-login
Oct 22 08:04:27 server kernel: [ 1267]   494  1267    11319      153   0       0             0 imap-login
Oct 22 08:04:27 server kernel: [ 1268]   494  1268    11319      153   0       0             0 imap-login
Oct 22 08:04:27 server kernel: [ 1269]   494  1269    11319      153   0       0             0 imap-login
Oct 22 08:04:27 server kernel: [ 1275]   494  1275    11319      153   0       0             0 imap-login
Oct 22 08:04:27 server kernel: [ 1276]   494  1276    11319      153   0       0             0 imap-login
Oct 22 08:04:27 server kernel: [ 1277]   494  1277    11319      153   0       0             0 imap-login
Oct 22 08:04:27 server kernel: [ 1278]   494  1278    11319      153   0       0             0 imap-login
Oct 22 08:04:27 server kernel: [ 1279]   494  1279    11319      153   0       0             0 imap-login
Oct 22 08:04:27 server kernel: [ 1280]     0  1280     4932      148   0       0             0 config
Oct 22 08:04:27 server kernel: [ 1281]     0  1281    16654	82   0     -17         -1000 sshd
Oct 22 08:04:27 server kernel: [ 1287]     0  1287     5518      148   0       0             0 auth
Oct 22 08:04:27 server kernel: [ 1300]     0  1300   130296      183   0       0             0 clamd
Oct 22 08:04:27 server kernel: [ 1309]   493  1309    17538      180   0       0             0 freshclam
Oct 22 08:04:27 server kernel: [ 1320]     0  1320    27076      185   0       0             0 mysqld_safe
Oct 22 08:04:27 server kernel: [ 1347]     0  1347     4775      102   0       0             0 da-popb4smtp
Oct 22 08:04:27 server kernel: [ 1433]   496  1433   221966     5626   0       0             0 mysqld
Oct 22 08:04:27 server kernel: [ 1347]     0  1347     4775      102   0       0             0 da-popb4smtp
Oct 22 08:04:27 server kernel: [ 1433]   496  1433   221966     5626   0       0             0 mysqld
Oct 22 08:04:27 server kernel: [ 1456]     8  1456    15402      110   0       0             0 exim
Oct 22 08:04:27 server kernel: [ 1463]     0  1463    60650      515   0       0             0 spamd
Oct 22 08:04:27 server kernel: [ 1472]     0  1472    62169      266   0       0             0 spamd
Oct 22 08:04:27 server kernel: [ 1497]     0  1497    29325      148   0       0             0 crond
Oct 22 08:04:27 server kernel: [ 1513]     0  1513    47840      271   0       0             0 python
Oct 22 08:04:27 server kernel: [ 1517]     0  1517     3438      176   0       0             0 qemu-ga
Oct 22 08:04:27 server kernel: [ 1525]     0  1525     1016      122   0       0             0 mingetty
Oct 22 08:04:27 server kernel: [ 1527]     0  1527     1016      122   0       0             0 mingetty
Oct 22 08:04:27 server kernel: [ 1529]     0  1529     1016      122   0       0             0 mingetty
Oct 22 08:04:27 server kernel: [ 1531]     0  1531     1016      122   0       0             0 mingetty
Oct 22 08:04:27 server kernel: [ 1533]     0  1533     1016      122   0       0             0 mingetty
Oct 22 08:04:27 server kernel: [ 1535]     0  1535     1016      122   0       0             0 mingetty
Oct 22 08:04:27 server kernel: [ 1543]     0  1543     2731	53   0     -17         -1000 udevd
Oct 22 08:04:27 server kernel: [ 1544]     0  1544     2731	53   0     -17         -1000 udevd
Oct 22 08:04:27 server kernel: [ 1572]    25  1572    40586      296   0       0             0 named
Oct 22 08:04:27 server kernel: [26844]    38 26844     7680      197   0       0             0 ntpd
Oct 22 08:04:27 server kernel: [16894]     0 16894    60650      204   0       0             0 spamd
Oct 22 08:04:27 server kernel: [24501]     0 24501    13802      124   0       0             0 directadmin
Oct 22 08:04:27 server kernel: [ 4311]   494  4311    11318      154   0       0             0 pop3-login
Oct 22 08:04:27 server kernel: [ 4676]   494  4676    11318      154   0       0             0 pop3-login
Oct 22 08:04:27 server kernel: [ 4681]   494  4681    11318      154   0       0             0 pop3-login
Oct 22 08:04:27 server kernel: [10326]   494 10326    11318      154   0       0             0 pop3-login
Oct 22 08:04:27 server kernel: [10327]   494 10327    11318      154   0       0             0 pop3-login
Oct 22 08:04:27 server kernel: [10328]   494 10328    11318      154   0       0             0 pop3-login
Oct 22 08:04:27 server kernel: [16927]   494 16927    11318      154   0       0             0 pop3-login
Oct 22 08:04:27 server kernel: [16928]   494 16928    11318      154   0       0             0 pop3-login
Oct 22 08:04:27 server kernel: [16929]   494 16929    11318      154   0       0             0 pop3-login
Oct 22 08:04:27 server kernel: [16930]   494 16930    11318      154   0       0             0 pop3-login
Oct 22 08:04:27 server kernel: [16931]   494 16931    11318      154   0       0             0 pop3-login
Oct 22 08:04:27 server kernel: [16932]   494 16932    11318      154   0       0             0 pop3-login
Oct 22 08:04:27 server kernel: [16934]   494 16934    11318      154   0       0             0 pop3-login
Oct 22 08:04:27 server kernel: [16935]   494 16935    11318      154   0       0             0 pop3-login
Oct 22 08:04:27 server kernel: [17032]   494 17032    11318      154   0       0             0 pop3-login
Oct 22 08:04:27 server kernel: [17033]   494 17033    11318      154   0       0             0 pop3-login
Oct 22 08:04:27 server kernel: [ 3364]     0  3364    13802	27   0       0             0 directadmin
Oct 22 08:04:27 server kernel: [ 3365]     0  3365    13802	27   0       0             0 directadmin
Oct 22 08:04:27 server kernel: [ 3366]     0  3366    13802	27   0       0             0 directadmin
Oct 22 08:04:27 server kernel: [ 3369]     0  3369    13802	27   0       0             0 directadmin
Oct 22 08:04:27 server kernel: [ 3386]     0  3386    13802	27   0       0             0 directadmin
Oct 22 08:04:27 server kernel: [ 3391]     0  3391    13802	27   0       0             0 directadmin
Oct 22 08:04:27 server kernel: [ 3400]     0  3400    13802	27   0       0             0 directadmin
Oct 22 08:04:27 server kernel: [ 3401]     0  3401    13802	27   0       0             0 directadmin
Oct 22 08:04:27 server kernel: [ 3402]     0  3402    13802	27   0       0             0 directadmin
Oct 22 08:04:27 server kernel: [ 3403]     0  3403    13802	27   0       0             0 directadmin
Oct 22 08:04:27 server kernel: [14000]     0 14000    40262     1020   0       0             0 lfd
Oct 22 08:04:27 server kernel: [14693]     0 14693     4074     1001   0       0             0 atop
Oct 22 08:04:27 server kernel: [26887]     0 26887    53607      374   0       0             0 httpd
Oct 22 08:04:27 server kernel: [26892]   503 26892    70435     1845   0       0             0 httpd
Oct 22 08:04:27 server kernel: [26893]   503 26893    61855     2217   0       0             0 httpd
Oct 22 08:04:27 server kernel: [26894]   503 26894    70862     1123   0       0             0 httpd
Oct 22 08:04:27 server kernel: [26895]   503 26895    68245     2591   0       0             0 httpd
Oct 22 08:04:27 server kernel: [27115]   503 27115    70761     2266   0       0             0 httpd
Oct 22 08:04:27 server kernel: [27118]   503 27118    70732     2865   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12383]   503 12383    65625     1870   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12385]   503 12385    67725     1601   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12388]   503 12388    65038     2423   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12389]   503 12389    65042     2102   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12390]   503 12390    67726     1571   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12391]   503 12391    67674     2084   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12397]   503 12397    65674     1132   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12398]   503 12398    65625     2708   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12399]   503 12399    65627     1296   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12400]   503 12400    65670     1764   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12401]   503 12401    65625     1619   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12402]   503 12402    65625     2676   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12403]   503 12403    66311     2281   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12404]   503 12404    65625     1798   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12405]   503 12405    65619      977   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12406]   503 12406    65621     1113   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12407]   503 12407    67087     1995   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12408]   503 12408    67138     1428   0       0             0 httpd
Oct 22 08:04:27 server kernel: [12409]   503 12409    65617     1402   0       0             0 httpd
OOct 22 08:04:47 server kernel: Out of memory: Kill process 1300 (clamd) score 49 or sacrifice child
Oct 22 08:04:47 server kernel: Killed process 1300, UID 0, (clamd) total-vm:521184kB, anon-rss:112kB, file-rss:620kB
Oct 22 08:04:47 server kernel: lfd invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
Oct 22 08:04:47 server kernel: lfd cpuset=/ mems_allowed=0
Oct 22 08:04:48 server kernel: Pid: 12683, comm: lfd Not tainted 2.6.32-431.29.2.el6.x86_64 #1
Oct 22 08:04:48 server kernel: Call Trace:
Oct 22 08:04:48 server kernel: [<ffffffff810d0361>] ? cpuset_print_task_mems_allowed+0x91/0xb0
Oct 22 08:04:48 server kernel: [<ffffffff81122730>] ? dump_header+0x90/0x1b0
Oct 22 08:04:48 server kernel: [<ffffffff812282fc>] ? security_real_capable_noaudit+0x3c/0x70
Oct 22 08:04:48 server kernel: [<ffffffff81122bb2>] ? oom_kill_process+0x82/0x2a0
Oct 22 08:04:48 server kernel: [<ffffffff81122af1>] ? select_bad_process+0xe1/0x120
Oct 22 08:04:48 server kernel: [<ffffffff81122ff0>] ? out_of_memory+0x220/0x3c0
Oct 22 08:04:48 server kernel: [<ffffffff8112f90f>] ? __alloc_pages_nodemask+0x89f/0x8d0
Oct 22 08:04:48 server kernel: [<ffffffff8116799a>] ? alloc_pages_vma+0x9a/0x150
Oct 22 08:04:48 server kernel: [<ffffffff8114958d>] ? do_wp_page+0xfd/0x920
Oct 22 08:04:48 server kernel: [<ffffffff8115c393>] ? swap_info_get+0x63/0xe0
Oct 22 08:04:48 server kernel: [<ffffffff8114a5ad>] ? handle_pte_fault+0x2cd/0xb00
Oct 22 08:04:48 server kernel: [<ffffffff8114b00a>] ? handle_mm_fault+0x22a/0x300
Oct 22 08:04:48 server kernel: [<ffffffff8104a8d8>] ? __do_page_fault+0x138/0x480
Oct 22 08:04:48 server kernel: [<ffffffff81069290>] ? pick_next_task_fair+0xd0/0x130
Oct 22 08:04:48 server kernel: [<ffffffff81528bc6>] ? schedule+0x176/0x3b0
Oct 22 08:04:48 server kernel: [<ffffffff8100bb8e>] ? apic_timer_interrupt+0xe/0x20
Oct 22 08:04:48 server kernel: [<ffffffff8152e99e>] ? do_page_fault+0x3e/0xa0
Oct 22 08:04:48 server kernel: [<ffffffff8152bd55>] ? page_fault+0x25/0x30
Oct 22 08:04:48 server kernel: Mem-Info:
Oct 22 08:04:48 server kernel: Node 0 DMA per-cpu:
Oct 22 08:04:48 server kernel: CPU    0: hi:    0, btch:   1 usd:   0
Oct 22 08:04:48 server kernel: Node 0 DMA32 per-cpu:
Oct 22 08:04:48 server kernel: CPU    0: hi:  186, btch:  31 usd:  30
Oct 22 08:04:48 server kernel: active_anon:105245 inactive_anon:107084 isolated_anon:2816
Oct 22 08:04:48 server kernel: active_file:193 inactive_file:298 isolated_file:0
Oct 22 08:04:48 server kernel: unevictable:1000 dirty:0 writeback:237 unstable:0
Oct 22 08:04:48 server kernel: free:12249 slab_reclaimable:2674 slab_unreclaimable:8197
.
.
.
 
Hello,

If your load is high, type:
Code: 
top
to track all current processes.
Whatever is at the top of the list, would be using the most CPU.
It would only count if you're viewing it during the moments of high load.

In DA itself, if you're getting the high load notices, DA should be including the output from top for the top processes.

I'm not convinced that the named messages are the cause.

I'm not sure about lfd though.. I'm not familiar with it, so not sure of it's errors.
However, this did catch my eye:
Code: 
[COLOR=#333333]lfserverd invoked oom-killer[/COLOR]
If your server is indeed running out of memory (oom), it would then use "swap" for it's over-memory usage.
Swap uses the disk and is incredibly slow, and could also explain the high load.

John
 
Hello Administrator

Thanks for responding. At present there are no heavy loads on my CPU. Every thing seems to be functioning normally. And the load I mentioned above was mainly on Read/Write. It had a 100GB I/O usage, indeed.

My VPS owns 1mb of Ram and I have only a website with not so many visitors at present as I have only started to make it public very recently. So I think logically I do not need to be worried about Ram. The Swap file, however, as you have mentioned, is drastically increased in size. Now I'm wondering what caused that sudden spike. Indeed my website seems to have had no visitors that day.

I already scanned my server with Clamav and Maldetect apps and except for a malware found by the latter, my server did not seem to have had any viruses, etc.

Would you please let me know what I can do to determine what cause that problem? I'll be glad if I can supply you with more information in this regard.

Best
Saeed
 
As far as i can tell, the kernel decided to kill "clamd" because it was using too much memory.
Code: 
Out of memory: Kill process 1300 (clamd) score 49 or sacrifice child
Oct 22 08:04:47 server kernel:[COLOR=#ff0000][B] Killed process 1300, UID 0, (clamd)[/B][/COLOR] [COLOR=#0000ff][B]total-vm:521184kB[/B][/COLOR], anon-rss:112kB, file-rss:620kB
Oct 22 08:04:47 server kernel: [COLOR=#800080][B]lfd invoked oom-killer[/B][/COLOR]: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
"clamd" was using 521184 kB of memory which seems to be more than the physical memory.
This is where "oom-killer" comes in and kills processes so that the server won't crash (lfd invoked oom-killer)
 
@Themis

Hi, Thanks for replying. You are quite right. But I really need to know what originally instigated this heavy memory consumption. That can help me to rectify the issue or take predictive measures. I can provide more lines of my log files if any one needs more information.

Regards
Saeed
 
Last edited:
You must log in or register to reply here.
Back
Top