Cannot restart Apache

I have a problem with Apache. Yesterday Apache didn't restart at midnight (when Apache restarts automaticly). The reason was a process that didn't stop. The error that I got was:

(48)Address already in use: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
Unable to open logs

Click to expand...

With 'ps aux' I found the following processes:

apache 16864 99.0 0.4 4812 4108 ?? R 4:55PM 925:27.17 /usr/local/apache/bin/httpd -DSSL (perl5.8.6)
apache 16802 0.0 0.4 4816 4104 ?? S 4:55PM 0:02.56 /usr/local/apache/bin/httpd -DSSL (perl5.8.6)

Click to expand...

Killing the PIDs didn't work, but kill them with the '-9' option did. After that I could restart Apache.

Last night the problem came back again. At midnight, Aoache didn't restart again.

In 'top' I found the following information:

16864 apache 118 0 4812K 4108K CPU1 1 925:11 98.93% 98.93% perl5.8.6

Click to expand...

The load of the server was 1.00 exactly over 1, 5 and 10 minutes.

Does anybody know what I can do to solve this? Big chance that Apache cannot restart tonight again.

Thanks in advance for your help!!

Rick

Hey,

You may give this a try:

http://help.directadmin.com/item.php?id=31

David

Hey David,

Thanks for your reply.

I already updated the bootscript, but without any result. The problem is that those processes will not stop using the normal commands. Only the option '-9' can stop them.

I tried to restart Apache and I didn't get an error, but I don't feel really confident with that. Does anybody have another solution?

Thanks a lot in advance!

apache 1 or 2 ?
check /var/log/messages

Wael

I am using Apache 2.

The last messages before Apache went down the first time were:

Mar 15 00:00:02 s003 proftpd[641]: <myserver> - received SIGHUP -- master server rehashing configuration file
Mar 15 00:40:05 s003 kernel: Limiting closed port RST response from 623 to 200 packets/sec
Mar 15 00:40:11 s003 kernel: Limiting closed port RST response from 412 to 200 packets/sec
Mar 15 00:40:23 s003 kernel: Limiting closed port RST response from 225 to 200 packets/sec

Click to expand...

There are no unknown messages in the logs, only things like:

Mar 14 20:35:46 s003 kernel: pid 84580 (httpd), uid 1003: exited on signal 6
Mar 14 20:35:46 s003 kernel: pid 84434 (httpd), uid 1003: exited on signal 6
Mar 14 20:40:00 s003 kernel: pid 84771 (httpd), uid 1003: exited on signal 6
Mar 14 20:40:01 s003 kernel: pid 84969 (httpd), uid 1003: exited on signal 6
Mar 14 20:40:31 s003 kernel: pid 84862 (httpd), uid 1003: exited on signal 6
Mar 14 20:40:31 s003 kernel: pid 84776 (httpd), uid 1003: exited on signal 6
Mar 14 20:45:23 s003 kernel: pid 84773 (httpd), uid 1003: exited on signal 6
Mar 14 20:45:23 s003 kernel: pid 85053 (httpd), uid 1003: exited on signal 6
Mar 14 20:45:23 s003 kernel: pid 85076 (httpd), uid 1003: exited on signal 6
Mar 14 20:45:24 s003 kernel: pid 85079 (httpd), uid 1003: exited on signal 6

Click to expand...

One of our clients using Apache 2 is having the same problem.

We also found it necessary to use the -9 option to stop apache. I'm considering that we may have to add this to the new apache start/stop script.

John, what do you think?

Jeff

I'd say only add -9 if you are affected by the problem.
Apache says that only the main httpd pid (/var/run/httpd.pid) process should be given the shut down notice, not the child processes. The parent should be the one doing that. If you kill the parent with a -9, then it won't be able to kill it's children as it's already dead. On linux system, the "killproc" will handle that... it's an effective child killer, but for freebsd/debian systems, it may cause problems.

So, it depends on the OS you're using. Give it a try to see if works for you. If you're not affected by the problem, I wouldn't recommend it.

John

kill -9 works on RHEL/CentOS; we've tested it.

We're writing a patch to /etc/rc.d/init.d/apache to do the following for a restart:

1) After it restarts apache, it checks to see if it's running.

If it's not, it looks to see if there's only the ONE line returned by ps waux | grep apache.

If there is, it'll do a kill -9 on the PID of that one line.

Then it'll do a restart.

That seems fairly safe to me, but it'll only work if DA uses the standard scripts.

John, does DA use the standard script at /etc/rc.d/init.d/apache?

Jeff

You mean apachectl? .. no, because our setup of apache 1.3 uses many if-def's that need to be setup at the command line.
If you want to use apachectl, you can, but have to manually set which modules you want loaded by removing the ifDefine options for the required modules.

Our Apache 2 setup (httpd.conf) doesn't use the ifDefine's, so you should be able to use apachectl out of the box.

John

No actually I was referring to the /etc/rc.d/init.d/httpd file. Is that what you use? Or do you use something else?

If you use that file it's easy enough for me to stick changes in there.

If you don't use that, then do you do something in C ? Or something else?

Thanks.

Jeff

We're seeing this happen at odd times ... not just at the midnight restart; I'm guessing it can happen any time DA restarts Apache.

I have a tested script that works to "fix" the problem and can be installed in the /etc/rc.d/init.d/apache restart command, but it'll only work if that's how DA restarts apache2.

So I'm awaiting a response from John before posting it.

Jeff

For Redhat/Fedora machines, DA will call: Debian: FreeBSD: During log rotation/tally, DA will take the PID from /var/run/httpd.pid and issue an HUP to the master apache process to reopen the logs after DA has renamed them all with a .1 extension.

The -9 should be fine... "kill" with no options defaults to TERM which is in fact -9 anyway:
http://httpd.apache.org/docs/2.0/stopping.html
so it looks like the parent process isn't killed instantly with a -9, it quickly nukes its kids when it gets a -9/TERM, so that should be fine. But if you do a "man kill" or "man killall", you'll note that the default is set to TERM anyway, so adding -9 isn't different from the default.

On linux systems, often times the "killproc" funciton is used. This may need to be replaced with the basic -9/TERM .. as the killproc likely goes to all children and kills them without the parent knowing (which is against what the documentation at apache.org says).. which may be the problem.

If you're using apache 2... the fix is probably as simple as removing the httpd boot script, and creating a link to the apachectl script.

John

John, the problems we're having is when DA starts/stops.

Presuming everything works as you say (of course I believe you ) our script should solve the problem. So far, we've not had any failures. Or if we have, they've been fixed by our change to the script.

If we replace the httpd boot script with a link to the apachectl script will apache start on system startup? Does apachectl, when passed start, know to start apache2?

Thanks.

Jeff

now that you mention it, apachectl needs the "startssl" option instead of "start", so no, it wouldn't work out of the box. I guess I was thinking of using the apachectl script, but it would have to be called from the current boot script ... so all work is done by apachectl, but the httpd script calls the startssl/stop/graceful options in apachectl. It doesn't really matter, they're fairly similar in what signals are passed to the httpd process anyway.

John

I'm glad I asked.

anyone know what RHEL uses in their startup script in a new install? They do use Apache2.

Jeff

jlasman said:

I have a tested script that works to "fix" the problem and can be installed in the /etc/rc.d/init.d/apache restart command, but it'll only work if that's how DA restarts apache2.

Click to expand...

Our first try at a script didn't help; we're testing our second attempt.

Jeff