Multiple Pids Shutting Down Server

[wildun]

All of my services are creating pids until the server stops working suddenly. When I reboot the server it starts all over again. This was in the admin messages.

This is an automated message notifying you that the 1 minute load average on your system is 52.43.
This has exceeded the 10 threshold.

One Minute - 52.43
Five Minutes - 14.55
Fifteen Minutes - 5.4

top - 12:30:02 up 8 min, 1 user, load average: 52.43, 14.55, 5.40
Tasks: 1439 total, 8 running, 1431 sleeping, 0 stopped, 0 zombie
Cpu(s): 16.3%us, 14.1%sy, 0.0%ni, 62.8%id, 6.3%wa, 0.0%hi, 0.4%si, 0.0%st
Mem: 9094852k total, 1735128k used, 7359724k free, 94580k buffers
Swap: 5931000k total, 0k used, 5931000k free, 530712k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3015 apache 16 0 48112 30m 4292 R 54.9 0.3 0:10.48 httpd
9595 root 21 0 3212 1732 692 R 18.8 0.0 0:00.25 top
462 root 10 -5 0 0 0 S 6.3 0.0 0:11.72 kjournald
2574 mysql 15 0 35204 15m 3208 S 1.6 0.2 0:07.01 mysqld
7088 mail 16 0 11768 1608 876 S 1.6 0.0 0:00.02 exim
7357 mail 16 0 11780 1608 876 S 1.6 0.0 0:00.01 exim
7590 mail 16 0 11780 1616 872 S 1.6 0.0 0:00.01 exim
7788 mail 16 0 11780 1608 872 S 1.6 0.0 0:00.01 exim
7841 mail 16 0 11780 1608 872 S 1.6 0.0 0:00.01 exim
8201 mail 16 0 11780 1616 876 S 1.6 0.0 0:00.02 exim
8236 mail 16 0 11776 1604 876 S 1.6 0.0 0:00.01 exim
8512 mail 16 0 11776 1604 876 S 1.6 0.0 0:00.01 exim
8721 mail 16 0 11780 1608 872 S 1.6 0.0 0:00.01 exim
9026 mail 15 0 11784 1612 872 D 1.6 0.0 0:00.01 exim
9092 mail 16 0 11780 1608 876 S 1.6 0.0 0:00.01 exim
9117 mail 16 0 11784 1612 876 S 1.6 0.0 0:00.01 exim
9394 mail 16 0 11776 1604 872 S 1.6 0.0 0:00.01 exim
9402 mail 16 0 11776 1604 876 S 1.6 0.0 0:00.01 exim
1 root 15 0 2152 628 540 S 0.0 0.0 0:01.70 init
2 root RT -5 0 0 0 S 0.0 0.0 0:00.08 migration/0
3 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
5 root RT -5 0 0 0 S 0.0 0.0 0:00.01 migration/1

Any advice will be appreciated. Not sure what to do here.

Think I found the issue but I have a question. If the server becomes inaccessable will this cause pids to continue to start indefinitely?

 

[nobaloney]

PID stands for Process ID, and whenver a process is tarted it gets a PID.

Looking at what you posted, you're not using any swap memory, and there aren't a large number of PIDs for any of the programs I see in the output.

You'll need to find out which programs are creating multiple PIDs (processes) and then begin to check logs to determine why.

Jeff

 

[tomtom901]

Also, the load is kinda high and Apache and Exim are eating

 

[nobaloney]

I missed that most of the problem appears to be in apache; exim doesn't seem to be using much.

I'd say start searching your httpd logs.

Jeff

 

[tomtom901]

I missed that most of the problem appears to be in apache; exim doesn't seem to be using much.

I'd say start searching your httpd logs.

Jeff

It states 15 Exim processes with a total CPU usage of 16%, that's a bit high don't you think Jeff?

 

[wildun]

Actually that was a much lower batch of pids than what were causing the issue. Exim was the big hog and it was being caused by an automated script registering spammers in wordpress on one domain. That was just the warning email in the message system. I do have swap memory.

The server has 10 gigs of physical memory and 5 gigs of swap. Two 2.6 processors and two 2.6 virtual processors. There are only 35 domains on it with two that have traffic of about a 1000 visitors a day each.

Once I disabled registrations on that site in wordpress things went back to normal, happened to be one of mine. That was the most recent WP so there must be security hole in it. The registration pages also uses captcha but the script was getting by it somehow.