troubleshoot Apache/system load

D

divinelighting

Verified User
Joined
Mar 17, 2008
Messages
108
Every 2 weeks or so, Apache stops responding and i have to restart it. Here is the ticket I get. I am the only user on the server and I can't imagine what could cause any substantial load. Any ideas?

Subject: Warning: The system load average is 11.06 Today at 06:42

This is an automated message notifying you that the 5 minute load average on your system is 11.06.
This has exceeded the 10 threshold.

One Minute - 19.68
Five Minutes - 11.06
Fifteen Minutes - 4.58

top - 06:42:02 up 59 days, 12:42, 0 users, load average: 19.68, 11.06, 4.58
Tasks: 197 total, 14 running, 183 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.1%us, 0.6%sy, 0.8%ni, 98.4%id, 0.1%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3367788k total, 3103432k used, 264356k free, 155572k buffers
Swap: 2096472k total, 92k used, 2096380k free, 2202156k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
11656 apache 20 4 58460 11m 2992 R 84.8 0.3 0:00.43 /usr/sbin/httpd -k start -DSSL
9773 apache 29 4 63244 23m 12m R 21.7 0.7 0:18.83 /usr/sbin/httpd -k start -DSSL
9071 apache 25 4 62940 25m 15m R 15.8 0.8 0:08.32 /usr/sbin/httpd -k start -DSSL
11091 apache 29 4 62964 17m 7360 R 15.8 0.5 0:19.95 /usr/sbin/httpd -k start -DSSL
11095 apache 29 4 58460 11m 2992 R 15.8 0.3 0:14.56 /usr/sbin/httpd -k start -DSSL
11100 apache 29 4 58460 11m 2992 R 15.8 0.3 0:10.95 /usr/sbin/httpd -k start -DSSL
11104 apache 29 4 58460 11m 2992 R 15.8 0.3 0:06.15 /usr/sbin/httpd -k start -DSSL
11084 apache 29 4 62960 18m 7668 R 11.8 0.6 0:26.80 /usr/sbin/httpd -k start -DSSL
9775 apache 29 4 62924 18m 8420 R 2.0 0.6 0:29.43 /usr/sbin/httpd -k start -DSSL
1 root 15 0 2160 600 520 S 0.0 0.0 0:02.49 init [3]
2 root RT -5 0 0 0 S 0.0 0.0 0:02.87 [migration/0]
3 root 36 19 0 0 0 S 0.0 0.0 0:00.12 [ksoftirqd/0]
4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/0]
5 root RT -5 0 0 0 S 0.0 0.0 0:02.45 [migration/1]
6 root 39 19 0 0 0 R 0.0 0.0 0:00.21 [ksoftirqd/1]
7 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/1]
8 root 10 -5 0 0 0 S 0.0 0.0 0:00.39 [events/0]
9 root 10 -5 0 0 0 S 0.0 0.0 0:00.30 [events/1]
10 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 [khelper]
11 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 [kthread]
15 root 12 -5 0 0 0 S 0.0 0.0 0:00.24 [kblockd/0]
16 root 10 -5 0 0 0 S 0.0 0.0 0:00.42 [kblockd/1]
17 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 [kacpid]
Click to expand...
 
My server is very underutilized, except for this anomaly.

Processor 1: Intel Core2Duo E5200 Dual Core - 2.50Ghz, 2M Cache, 800FSB
Processor 2: Not Applicable
Memory: 4GB DDR RAM
RAM Type: SELECT FROM DROP DOWN
Storage Drive 1: 500GB SATA 7,200RPM
Click to expand...
 
Actually seems you have 3gb of ram and not 4gb.. So.. now my question is, what kernel and os are you using?

Cause maybe you using a 32Bit OS and not an SMP kernel. Am i wrong?

When the system load is so high the max load is from apache? Have you checked for low disk space in one partition and/or for any bad perl script (in tmp partition for exmaple)?


Regards
 
More Info:

Processor Name Pentium(R) Dual-Core CPU E5200 @ 2.50GHz
Vendor ID GenuineIntel
Processor Speed (MHz) 1200.000
Processor Name Pentium(R) Dual-Core CPU E5200 @ 2.50GHz
Vendor ID GenuineIntel
Processor Speed (MHz) 2500.000
Total Memory 3367788 kB
Free Memory 308716 kB
Total Swap Memory 2096472 kB
Free Swap Memory 2096380 kB
System Uptime 59 Days, 13 Hours and 39 Minutes
Apache 2.2.17 Running
DirectAdmin 1.38.3 Running
Exim 4.67 Running
MySQL 5.0.67 Running
Named 9.3.6 Running
ProFTPd 1.3.1 Running
sshd Running
dovecot 1.2.4 Running
Php 5.2.10 Installed
Click to expand...

Kernel 2.6.18-238.5.1.el5

There are no cron jobs running at that time, and no scripts that I know of.
 
Thanks for the suggestions.

Updated exim and directadmin. The rest are more complicated and will have to wait until I change servers later in the month.
 
It's very hard to find out, in real time, why this sort of problem occurs, since it's likely based on code use on heavily used, or poorly written, sites.

So we've created a script to help monitor it, and to fix it by restarting apache as necessary (which doesn't seem to cause problems).

Look here (nobaloney.net).

Jeff
 
Jlasman, you made my day.

Would you be so kind as to provide command lines for loading this?

Thanks!
 
I don't offer help for installing my scripts because I honestly feel that anyone who's not versed in at least elementary server administration should be working at the shell level; it's simply too dangerous, and I can't take responsibility for anything you do on your server.

That said:

Edit the script so it sends email to you as it should. If you don't know how to do that by reading the script you should give up now and hire someone to do the install and/or optionally manage your server for you.

Upload it to a reasonable location on your server (I use /root/scripts/).

Chown it to root:root.

Chmod it to 700.

Create a cron to run it as root every five minutes.

Jeff
 
You must log in or register to reply here.
Back
Top