Unable to serve *any* page on fresh Apache httpd 2.4.41 - process aborts

W

Willis

Verified User
Joined
Dec 31, 2005
Messages
38
Fresh DA install on CentOS 8 (thought about posting in that forum, but the only issue I'm having is with httpd).

No installed users yet, just trying to view http://[ip] to see the default page, and it fails.

Code: 
[Fri Oct 25 18:27:01.370075 2019] [ssl:warn] [pid 31011:tid 139754341652736] AH01909: www.example.com:443:0 server certificate does NOT include an ID which matches the server name
[Fri Oct 25 18:27:01.370236 2019] [suexec:notice] [pid 31011:tid 139754341652736] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Fri Oct 25 18:27:01.382712 2019] [ssl:warn] [pid 31011:tid 139754341652736] AH01909: www.example.com:443:0 server certificate does NOT include an ID which matches the server name
[Fri Oct 25 18:27:01.383014 2019] [lbmethod_heartbeat:notice] [pid 31011:tid 139754341652736] AH02282: No slotmem from mod_heartmonitor
[Fri Oct 25 18:27:01.383034 2019] [:notice] [pid 31011:tid 139754341652736] mod_ruid2/0.9.8 enabled
[Fri Oct 25 18:27:01.384659 2019] [mpm_event:notice] [pid 31011:tid 139754341652736] AH00489: Apache/2.4.41 (Unix) OpenSSL/1.1.1 configured -- resuming normal operations
[Fri Oct 25 18:27:01.384693 2019] [core:notice] [pid 31011:tid 139754341652736] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Fri Oct 25 18:29:47.560338 2019] [core:notice] [pid 31011:tid 139754341652736] AH00052: child pid 31013 exit signal Aborted (6)
[Fri Oct 25 18:29:47.560382 2019] [core:notice] [pid 31011:tid 139754341652736] AH00052: child pid 31015 exit signal Aborted (6)
[Fri Oct 25 18:29:49.562700 2019] [core:notice] [pid 31011:tid 139754341652736] AH00052: child pid 32083 exit signal Aborted (6)
[Fri Oct 25 18:29:51.565157 2019] [core:notice] [pid 31011:tid 139754341652736] AH00052: child pid 32247 exit signal Aborted (6)
[Fri Oct 25 18:29:53.567755 2019] [core:notice] [pid 31011:tid 139754341652736] AH00052: child pid 32411 exit signal Aborted (6)
[Fri Oct 25 18:29:55.570264 2019] [core:notice] [pid 31011:tid 139754341652736] AH00052: child pid 32575 exit signal Aborted (6)
[Fri Oct 25 18:29:57.572635 2019] [core:notice] [pid 31011:tid 139754341652736] AH00052: child pid 32747 exit signal Aborted (6)
[Fri Oct 25 18:29:59.574257 2019] [core:notice] [pid 31011:tid 139754341652736] AH00052: child pid 451 exit signal Aborted (6)
[Fri Oct 25 18:30:01.576808 2019] [core:notice] [pid 31011:tid 139754341652736] AH00052: child pid 629 exit signal Aborted (6)
[Fri Oct 25 18:30:03.579142 2019] [core:notice] [pid 31011:tid 139754341652736] AH00052: child pid 869 exit signal Aborted (6)

All 10 exit signal Aborted's are triggered from a single http request of /.

I tried ratcheting the LogLevel up to debug and trace6, and nothing seems out of the ordinary aside from the AH00052 errors I am getting at the default warn log level.

Configured with php-fpm/php 7.2.24, but the errors are occuring on simple static html files, not even attempting to utilize php.

apachectl -M:

Code: 
Loaded Modules:
 core_module (static)
 authn_file_module (static)
 authn_dbm_module (static)
 authn_anon_module (static)
 authn_dbd_module (static)
 authn_socache_module (static)
 authn_core_module (static)
 authz_host_module (static)
 authz_groupfile_module (static)
 authz_user_module (static)
 authz_dbm_module (static)
 authz_owner_module (static)
 authz_dbd_module (static)
 authz_core_module (static)
 access_compat_module (static)
 auth_basic_module (static)
 auth_form_module (static)
 auth_digest_module (static)
 allowmethods_module (static)
 file_cache_module (static)
 cache_module (static)
 cache_disk_module (static)
 cache_socache_module (static)
 socache_shmcb_module (static)
 socache_dbm_module (static)
 socache_memcache_module (static)
 socache_redis_module (static)
 so_module (static)
 watchdog_module (static)
 macro_module (static)
 dbd_module (static)
 dumpio_module (static)
 buffer_module (static)
 ratelimit_module (static)
 reqtimeout_module (static)
 ext_filter_module (static)
 request_module (static)
 include_module (static)
 filter_module (static)
 substitute_module (static)
 sed_module (static)
 deflate_module (static)
 http_module (static)
 mime_module (static)
 log_config_module (static)
 log_debug_module (static)
 logio_module (static)
 env_module (static)
 expires_module (static)
 headers_module (static)
 unique_id_module (static)
 setenvif_module (static)
 version_module (static)
 remoteip_module (static)
 proxy_module (static)
 proxy_connect_module (static)
 proxy_ftp_module (static)
 proxy_http_module (static)
 proxy_fcgi_module (static)
 proxy_scgi_module (static)
 proxy_uwsgi_module (static)
 proxy_fdpass_module (static)
 proxy_wstunnel_module (static)
 proxy_ajp_module (static)
 proxy_balancer_module (static)
 proxy_express_module (static)
 proxy_hcheck_module (static)
 session_module (static)
 session_cookie_module (static)
 session_dbd_module (static)
 slotmem_shm_module (static)
 ssl_module (static)
 http2_module (static)
 lbmethod_byrequests_module (static)
 lbmethod_bytraffic_module (static)
 lbmethod_bybusyness_module (static)
 lbmethod_heartbeat_module (static)
 unixd_module (static)
 dav_module (static)
 status_module (static)
 autoindex_module (static)
 info_module (static)
 suexec_module (static)
 cgi_module (static)
 dav_fs_module (static)
 dav_lock_module (static)
 vhost_alias_module (static)
 negotiation_module (static)
 dir_module (static)
 actions_module (static)
 speling_module (static)
 userdir_module (static)
 alias_module (static)
 rewrite_module (static)
 ruid2_module (shared)
 systemd_module (shared)
 mpm_event_module (shared)

I thought this might be a fluke, but this is two new CentOS 8 VMs from scratch, and they both had this issue out of the box. Installed from CentOS 8 netinstall on KVM.
Nothing I can identify as relevant hitting system message log.

Here is the full error log from trace6 modified to mask IP and removing redundant lines for spinning up http2:trace2 worker threads:

Code: 
[Fri Oct 25 17:14:37.457263 2019] [mpm_event:trace6] [pid 10650:tid 140066810627840] event.c(1632): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0)
[Fri Oct 25 17:14:37.457358 2019] [http2:trace1] [pid 10650:tid 140068394911488] h2_h2.c(588): [client 256.256.256.256:19482] h2_h2, process_conn
[Fri Oct 25 17:14:37.457369 2019] [http2:trace1] [pid 10650:tid 140068394911488] h2_h2.c(602): [client 256.256.256.256:19482] h2_h2, process_conn, new connection using protocol 'http/1.1', direct=1, tls acceptable=1
[Fri Oct 25 17:14:37.457398 2019] [http2:trace2] [pid 10650:tid 140068394911488] h2_h2.c(639): [client 256.256.256.256:19482] h2_h2, not detected in 24 bytes(base64): R0VUIC8gSFRUUC8xLjENCkhvc3Q6IDEw
[Fri Oct 25 17:14:37.457407 2019] [http2:trace1] [pid 10650:tid 140068394911488] h2_h2.c(661): [client 256.256.256.256:19482] h2_h2, declined
[Fri Oct 25 17:14:37.457438 2019] [core:trace5] [pid 10650:tid 140068394911488] protocol.c(710): [client 256.256.256.256:19482] Request received from client: GET / HTTP/1.1
[Fri Oct 25 17:14:38.249615 2019] [core:notice] [pid 8293:tid 140069324589312] AH00052: child pid 10650 exit signal Aborted (6)
[Fri Oct 25 17:14:38.249658 2019] [core:trace4] [pid 8293:tid 140069324589312] mpm_common.c(536): mpm child 10650 (gen 0/slot 0) exited
[Fri Oct 25 17:14:39.250698 2019] [mpm_event:trace5] [pid 8293:tid 140069324589312] event.c(2886): Spawning new child: slot 0 active / total daemons: 0/0
[Fri Oct 25 17:14:39.250945 2019] [core:trace4] [pid 8293:tid 140069324589312] mpm_common.c(536): mpm child 10908 (gen 0/slot 0) started
[Fri Oct 25 17:14:39.252426 2019] [watchdog:debug] [pid 10908:tid 140069324589312] mod_watchdog.c(590): AH02981: Watchdog: Created child worker thread (_proxy_hcheck_).
[Fri Oct 25 17:14:39.252491 2019] [proxy:debug] [pid 10908:tid 140069324589312] proxy_util.c(1935): AH00925: initializing worker proxy:reverse shared
[Fri Oct 25 17:14:39.252506 2019] [proxy:debug] [pid 10908:tid 140069324589312] proxy_util.c(1992): AH00927: initializing worker proxy:reverse local
[Fri Oct 25 17:14:39.252525 2019] [proxy:debug] [pid 10908:tid 140069324589312] proxy_util.c(2027): AH00930: initialized pool in child 10908 for (*) min=0 max=159 smax=159
[Fri Oct 25 17:14:39.252554 2019] [http2:trace3] [pid 10908:tid 140069324589312] h2_conn.c(135): h2_workers: min=64 max=96, mthrpchild=64, idle_secs=600
[Fri Oct 25 17:14:39.254475 2019] [mpm_event:debug] [pid 10908:tid 140068403304192] event.c(2315): AH02471: start_threads: Using epoll (wakeable)
[Fri Oct 25 17:14:39.256282 2019] [http2:trace1] [pid 10908:tid 140068394911488] h2_h2.c(588): [client 256.256.256.256:13494] h2_h2, process_conn
[Fri Oct 25 17:14:39.256306 2019] [http2:trace1] [pid 10908:tid 140068394911488] h2_h2.c(602): [client 256.256.256.256:13494] h2_h2, process_conn, new connection using protocol 'http/1.1', direct=1, tls acceptable=1
[Fri Oct 25 17:14:39.256332 2019] [http2:trace2] [pid 10908:tid 140068394911488] h2_h2.c(639): [client 256.256.256.256:13494] h2_h2, not detected in 24 bytes(base64): R0VUIC8gSFRUUC8xLjENCkhvc3Q6IDEw
[Fri Oct 25 17:14:39.256338 2019] [http2:trace1] [pid 10908:tid 140068394911488] h2_h2.c(661): [client 256.256.256.256:13494] h2_h2, declined
[Fri Oct 25 17:14:39.256361 2019] [core:trace5] [pid 10908:tid 140068394911488] protocol.c(710): [client 256.256.256.256:13494] Request received from client: GET / HTTP/1.1
[Fri Oct 25 17:14:40.252152 2019] [core:notice] [pid 8293:tid 140069324589312] AH00052: child pid 10908 exit signal Aborted (6)
[Fri Oct 25 17:14:40.252197 2019] [core:trace4] [pid 8293:tid 140069324589312] mpm_common.c(536): mpm child 10908 (gen 0/slot 0) exited
[Fri Oct 25 17:14:41.253253 2019] [mpm_event:trace5] [pid 8293:tid 140069324589312] event.c(2886): Spawning new child: slot 0 active / total daemons: 0/0
[Fri Oct 25 17:14:41.253500 2019] [core:trace4] [pid 8293:tid 140069324589312] mpm_common.c(536): mpm child 11072 (gen 0/slot 0) started
[Fri Oct 25 17:14:41.255721 2019] [watchdog:debug] [pid 11072:tid 140069324589312] mod_watchdog.c(590): AH02981: Watchdog: Created child worker thread (_proxy_hcheck_).
[Fri Oct 25 17:14:41.255817 2019] [proxy:debug] [pid 11072:tid 140069324589312] proxy_util.c(1935): AH00925: initializing worker proxy:reverse shared
[Fri Oct 25 17:14:41.255836 2019] [proxy:debug] [pid 11072:tid 140069324589312] proxy_util.c(1992): AH00927: initializing worker proxy:reverse local
[Fri Oct 25 17:14:41.255858 2019] [proxy:debug] [pid 11072:tid 140069324589312] proxy_util.c(2027): AH00930: initialized pool in child 11072 for (*) min=0 max=159 smax=159
[Fri Oct 25 17:14:41.255888 2019] [http2:trace3] [pid 11072:tid 140069324589312] h2_conn.c(135): h2_workers: min=64 max=96, mthrpchild=64, idle_secs=600
[Fri Oct 25 17:14:41.259163 2019] [mpm_event:debug] [pid 11072:tid 140068403304192] event.c(2315): AH02471: start_threads: Using epoll (wakeable)
[Fri Oct 25 17:14:41.261259 2019] [http2:trace1] [pid 11072:tid 140068394911488] h2_h2.c(588): [client 256.256.256.256:34582] h2_h2, process_conn
[Fri Oct 25 17:14:41.261288 2019] [http2:trace1] [pid 11072:tid 140068394911488] h2_h2.c(602): [client 256.256.256.256:34582] h2_h2, process_conn, new connection using protocol 'http/1.1', direct=1, tls acceptable=1
[Fri Oct 25 17:14:41.261321 2019] [http2:trace2] [pid 11072:tid 140068394911488] h2_h2.c(639): [client 256.256.256.256:34582] h2_h2, not detected in 24 bytes(base64): R0VUIC8gSFRUUC8xLjENCkhvc3Q6IDEw
[Fri Oct 25 17:14:41.261331 2019] [http2:trace1] [pid 11072:tid 140068394911488] h2_h2.c(661): [client 256.256.256.256:34582] h2_h2, declined
[Fri Oct 25 17:14:41.261477 2019] [core:trace5] [pid 11072:tid 140068394911488] protocol.c(710): [client 256.256.256.256:34582] Request received from client: GET / HTTP/1.1
[Fri Oct 25 17:14:42.254606 2019] [core:notice] [pid 8293:tid 140069324589312] AH00052: child pid 11072 exit signal Aborted (6)
[Fri Oct 25 17:14:42.254666 2019] [core:trace4] [pid 8293:tid 140069324589312] mpm_common.c(536): mpm child 11072 (gen 0/slot 0) exited

Any thoughts or suggestions would be appreciated.

Edit:
Note, httpd compiled to mpm_prefork_module had the same issues as mpm_event_module.

Edit 2:
mod_ruid2 was the culprit (despite not giving any relevant errors) and was failing event with mod_php/mpm_prefork. Disabling mod_ruid2 in either mode resolves the issue.
 
Last edited:
A wild guess, you have used a minimal centos 8 install without any LAMP stack ?
 
Correct, I have used a CentOS 8 netinstall using server-only. Apache httpd was installed fully by DA in ./setup.sh auto
Apache httpd is running, but the worker processes are aborting.

Install process was:
Install CentOS 8 netinstall server only on KVM.
Install CentOS 8 pre-reqs for DA.
Install DA via ./setup.sh auto
After successful install, attempt to visit http://[ip] and see failures in error log.

Rebuilding httpd through custom build appears to show no issues, and continues to fail in the same way.
 
I've just re-issued a custom build via ./build apache and the log is attached just for better reference. I see no relevant errors in the build log.
 

Attachments

My first install was without auto, the last two have been with. All 3 have ended with the same issue.
Yes, I have read the install guide and meet all the requirements very easily.
 
mod_ruid2 is disabled ? and try php 7.3 instead of 7.2 as test
 
mod_ruid2 is enabled by default and shows no error in the apache httpd build process (you can find it building mod_ruid2 in the build log I attached earlier). On my first install, I did in fact try both 7.2 and 7.3, both give the same error. I'm simply trying to serve the static index.html from /var/www/html/index.html and that is failing.

I will try compiling with mod_ruid2 disabled.
 
Bingo, I am able to get a static file with mod_ruid2 disabled. Now the question is why there is no error about mod_ruid2.
 
mod_ruid2 works not good with php-fpm , and it has no use anymore.
Its advised not to use this combination
 
I was getting the same error with mod_ruid2 enabled and mpm-prefork/mod_php. DA defaults to mod_ruid2 enabled and mpm-prefork/mod_php.
 
Yes mod_php and mod_ruid2 works ok but if you switch to php-fpm you must disable mod_ruid2
 
No, I was getting the same process aborts on mod_php with mod_ruid2.

Like I said, ./setup.sh auto results in http 2.4 with mod_ruid2 and php_mod/mpm_prefork, and it gives the same exact process abort error on CentOS 8 for me.

I am considering this matter here closed, and I am moving the mod_ruid2 discussion to the CentOS 8 forum.
 
You must log in or register to reply here.
Back
Top