Very High server load - why?

Recently installed redhat 9 with da, and spamassassin 3.0 installed. I get a load average over 2.0 with only one user.

When I reboot the load goes back to normal, but slowly increases to even 5.0!!

Uptime:
21:14:51 up 2:41, 1 user, load average: 2.03, 1.92, 1.52

System Information:
Linux ns3.artimedia.nl 2.6.8-1.521 #1 Mon Aug 16 09:01:18 EDT 2004 i686 i686 i386 GNU/Linux


Memory Usage (MB):
total used free shared buffers cached
Mem: 979 339 640 0 58 109
-/+ buffers/cache: 172 807
Swap: 1976 0 1976


Disk Usage:
Filesystem Size Used Avail Use% Mounted on
/dev/hdb1 130G 4.8G 119G 4% /
/dev/hda1 99M 7.6M 87M 9% /boot
none 490M 0 490M 0% /dev/shm
/dev/hda6 127G 1.9G 118G 2% /home
/dev/hda5 4.9G 33M 4.6G 1% /tmp
/dev/hda3 9.7G 2.2G 7.0G 24% /usr
/dev/hda2 9.7G 376M 8.8G 5% /var
/dev/hdc1 151G 11G 133G 8% /backup


CPU Information:
processor : 0
model name : Intel(R) Pentium(R) 4 CPU 2.80GHz


I cant see to many processes running.
Anyone had an idea what can be causing this?


Apache httpd (pid 13586 ) Start Stop Restart Reload
DirectAdmin directadmin (pid 1378 13528 13530 13551 13565 13589 ) Start Stop Restart Reload
Exim exim (pid 1392 ) Start Stop Restart Reload
MySQL mysqld (pid 9926 9936 9937 9938 9939 9940 9966 9967 9968 9969 ) Start Stop
Named named (pid 1784 ) Start Stop Restart Reload
ProFTPd proftpd (pid 1450 ) Start Stop Restart
sshd sshd (pid 1347 4601 ) Start Stop Restart Reload
vm-Pop3d vm-pop3d (pid 12942 13358 13469 13470 13475 13476 13478 13479 13481 13501 13502 13529 )

Top

Thanks for that quick reply. Sorry I forgot the top output.

The load is now 6.0 (at 5 am in the morning, with no user connected). So it must be some server process going out of hand...

Looks here that spamd is causing the problem....

2054 tobe 25 0 28824 23m 6228 R 20.3 2.4 223:37.99 spamd
2055 tobe 25 0 29404 23m 6228 R 20.3 2.4 165:32.36 spamd
19787 tobe 25 0 32124 24m 6228 R 20.3 2.5 23:37.93 spamd
19801 tobe 25 0 29332 23m 6228 R 20.3 2.4 50:48.84 spamd
2052 tobe 25 0 29020 23m 6228 R 18.3 2.4 152:36.52 spamd
26317 root 16 0 2812 960 1620 R 0.3 0.1 0:00.28 top
1 root 16 0 3100 460 1316 S 0.0 0.0 0:02.82 init
2 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
3 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 events/0
4 root 15 -10 0 0 0 S 0.0 0.0 0:00.00 khelper
5 root 15 -10


So it seems that spamd is causing this, for the user tobe.

After a lot of looking around I discoverd this thread on another forum: read below -->

There was one user that was "over quota" and was still receiving a LOT of Spam E-mail (approximately 1000 a day). As soon as he was over quota, sendmail couldn't deliver the pending E-mails from the mailqueue anymore, and what seemed to happen was something like a "vicious circle" wherein SpamAssassin didn't stop scanning all of the E-mails waiting in the mailqueue. The only solution was to either upgrade the user's quota, or to manually delete the Spam E-mails waiting in the mqueue.

I noticed that this mentioned user tobe was indeed a little over quota , so it could be that this is causing the spamd vicious circle mentioned here.

Anyone has experienced this before?

Re: Top

I've never experienced it, and hopefully exim is smart enough to not check something again that's already been checked.

Of course it's possible.

We use SpamBlocker to keep spamd from having to deal with so much spam.

Jeff

where can i get spamblocker?

Well, it seems that it was indeed spamd running in circels, on that same user account. Because I raised the account, so that he was not over his limit anymore, restarted the server, and problem solved. Load is down to 0.10 and less and stable now.

But using spamblocker sounds interesting, where can i get it and can I install it myself?

thanks,

SpamBlocker is part of the exim.conf file that I wrote that should be installed with your copy of DA.

If your copy of exim.conf looks similar to

this one, then it's already installed.

Simply make sure that all the file changes and additions in the comments have been made, and turn it on by putting the domains you want spam blocked for in /etc/virtual/use_rbl_domains.

You should know that SpamBlocker is somewhat controversial, and that's why I left it off by default.

After I'm done adding VirusBlocker and have the new exim.conf file distributed I'll work on making sure SpamAssassin doesn't work in loops. It should be easily doable, but I just don't have the time now.

Jeff

problem still

Thanks again Jeff.

It appears that the problem first mentioned in this post has not been solved. Increasing the quota for this user does not help.

After restarting the mailserver, and than after a couple of hours,
the load always goes back to 5.0 or 6.0.

In top I see that same as posted earlier, 4 times spamd working on email for the same domain.

When I kill spamd and restart, the problem dissapears for couple of hours and than the load starts rising back up again.

Any ideas??

thanks

Have you installed SpamBlocker for that user's domain? If you do that it should cut down quite a bit of the spam coming in.

Jeff