Apache 1.3.41 with PHP 5.2.11 Segmentation Fault (11)

J

jw00dy

Verified User
Joined
Dec 6, 2004
Messages
191
Location
Utah, USA
For the last week, my apache setup (CentOS 5 64bit) has been having problems staying running for longer than a day. I am getting the following errors in my error_log

[Sun Nov 1 02:43:19 2009] [notice] Apache/1.3.41 (Unix) mod_perl/1.29 FrontPage/5.0.2.2510 mod_ssl/2.8.31 OpenSSL/0.9.8e-fips-rhel5 configured -- resuming normal operations
[Sun Nov 1 02:43:19 2009] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sun Nov 1 02:43:19 2009] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Sun Nov 1 02:46:16 2009] [notice] child pid 17437 exit signal Segmentation fault (11)
[Sun Nov 1 02:46:54 2009] [notice] child pid 17427 exit signal Segmentation fault (11)
[Sun Nov 1 02:47:28 2009] [notice] child pid 17421 exit signal Segmentation fault (11)
[Sun Nov 1 02:49:44 2009] [notice] child pid 17425 exit signal Segmentation fault (11)
Click to expand...

I need help figuring out what is causing the Segmentation fault and hopefully get this fixed.

Your help is appreciated.
 
Are there any PHP extensions installed? If yes, try disabling them.
 
I have xcache installed -- disabled it (I actually upgraded it as well) and it didn't seem to make a difference.

How can I tell what other (if any) extensions are installed? That is the only one I can see in php.ini.

And thanks for your help Martynas!
 
Well, after banging my head and then opening a support ticket, I've come a little further along. Still not fixed, but I have some debug info I got using gdb from the Apache core dump files.

Here is the lastest one -- this is just the end of the debug (a few lines before I issue where, and everything after it).

Reading symbols from /lib64/libnss_dns.so.2...done.
Loaded symbols for /lib64/libnss_dns.so.2

Core was generated by `/usr/sbin/httpd -DHAVE_STATUS -DHAVE_NEGOTIATION -DHAVE_SETENVIF -DHAVE_CERN_ME'.
Program terminated with signal 11, Segmentation fault.
[New process 12960]
#0 0x00002afbe8f804f2 in addbyter () from /usr/local/lib/libcurl.so.4
(gdb) where
#0 0x00002afbe8f804f2 in addbyter () from /usr/local/lib/libcurl.so.4
#1 0x00002afbe8f80dec in dprintf_formatf () from /usr/local/lib/libcurl.so.4
#2 0x00002afbe8f81e45 in curl_mvsnprintf () from /usr/local/lib/libcurl.so.4
#3 0x00002afbe8f806b3 in curl_msnprintf () from /usr/local/lib/libcurl.so.4
#4 0x00002afbe8f6fece in Curl_failf () from /usr/local/lib/libcurl.so.4
#5 0x00002afbe8f681c1 in Curl_resolv_timeout () from /usr/local/lib/libcurl.so.4
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Click to expand...

It would appear that libcurl is the problem or maybe libnss_dns.so.2 (which from what I can gather is a part of glibc.

Anyway, I'm still getting segmentation faults so any additional help would really be appreciated...
 
I'm having an hard time trying to debug Apache's FastCGI handler crashes myself on a quite complex setup, can you point me on the right direction please? How did you debug it?

I tried to attach an strace session to the offending child without success and the parent process doesn't crash so gdb won't be useful (will it?).

I can't modify the setup because of too much traffic, if I deactivate temporarily FastCGI or APC there is too much latency.

I'm quite sure this is a critical bug in some library or one of the experimental features I'm using in my setup, but I have updated most of them since (this problem began a couple of months ago) without a change.

Regarding your problem, maybe it's related... but I'm not on a x86_64 system.
 
Last edited:
Yeah, I'm on x86_64 as well. CentOS is my flavor.

I'm not sure about your question, as I didn't attach directly to httpd, I just enabled coredumps, and used gdb against the core dump files themselves. I did read about what you are trying to do and from the sounds of it, you have to stop apache and then start it with the -X (could be upper or lower case, but I don't recall) command to get it to work in single something or other mode, in which case you can run gdb against it.

On a different note, I did read a LOT of information about optimizers, and though I don't recall APC being one of them, I would look into maybe switching to xcache and see if that changes anything.

I do know that eaccelerator was one that DID have lots of problems.

Hope that helps.
 
I finally solved the problem I was having.

cURL was the problem, but it didn't go away until one of my clients resolved a problem with their expired SSL certificate. I'm not sure why this would affect cURL, but for some reason it did. Shortly after updating the certificate, the errors stopped and Apache has been running as it's supposed to since.
 
That's so weird, I can't find a reason for an expired SSL certificate (or even a broken one) to mess with the cURL library. Anyway, mine has just stopped crashing by itself, probably after one of the many system updates I've been doing. Hurray!
 
Honestly, I can't directly link it to the SSL cert either. I only know it stopped, and it stopped within a few hours of the update of the SSL cert. I was ignoring it at the time since I've banged my head trying to figure out why it wasn't working. The only thing I did that would have changed anything was update the SSL certificate.

I did have to create a new Self signed Cert with a 2048 bit length before I could get the CSR to then get the certificate from the company they bought it from, but that was all I did.

About 8 hours or so after that, the errors stopped, cURL started working and live's been good (previously it was failing in some wordpress stuff)

I suppose a YUM update that automatically ran might have done it, but I see nothing in the Yum.log file to indicate that.

I have compared the SSL install time (Nov 10, 20:54) and the point the SegFaults stopped and their is pretty close to exactly an 8 hour difference (Nov 11, 06:56).

So... your guess is as good as mine really.
 
You must log in or register to reply here.
Back
Top