Apache 2.4.66 - mod_http2 bug

[matkra]

Hello,

I am experiencing an issue with Apache after upgrading to version 2.4.66. Since the update, Apache has been crashing randomly under load.

Based on core dump analysis, the crashes are related to mod_http2. The backtrace clearly points to a double free / invalid free in APR pools during HTTP/2 stream cleanup.

This appears to be a known issue:

Bug 69899 – Double free / invalid free crash in mod_http2 on Apache 2.4.66

Log in to ASF Bugzilla

bz.apache.org

What is the recommended way to apply this patch ?

 

[fln]

We have already cherry-picked the patch for it for Apache 2.4.66. The issue is not trivial to reproduce, would you be able to test it out and report if it fixes the issue?

We have prepared a special DA build where Apache 2.4.66 has additional patch for this bug. To test it out:

1. Remove the custom Apache version (if customized) this makes sure you will use Apache 2.4.66:
sed -i -e '/^apache/d' /usr/local/directadmin/custombuild/custom_versions.txt

2. Update DA to version 1.691 build 7649164538936cfc25718c642745aa970a2bc285
da update 7649164538936cfc25718c642745aa970a2bc285

3. Recompile apache web server
da build apache

Please note that if you upgrade to a custom DA build, the auto-updater will revert DirectAdmin back to the default version after ~30 minutes. This means it would be best to start apache build immediately after updating DA to custom build.

A quick check to see if apache is patched or not is to test for mod_http2 version. Patched apache will have mod_http2 2.0.37, while the default apache will have 2.0.35. A quick test from CLI would be:
strings /usr/sbin/httpd | grep -q -F 2.0.37 && echo patched || echo non-patched

 

[matkra]

We have already cherry-picked the patch for it for Apache 2.4.66. The issue is not trivial to reproduce, would you be able to test it out and report if it fixes the issue?

We have prepared a special DA build where Apache 2.4.66 has additional patch for this bug. To test it out:

1. Remove the custom Apache version (if customized) this makes sure you will use Apache 2.4.66:
sed -i -e '/^apache/d' /usr/local/directadmin/custombuild/custom_versions.txt

2. Update DA to version 1.691 build 7649164538936cfc25718c642745aa970a2bc285
da update 7649164538936cfc25718c642745aa970a2bc285

3. Recompile apache web server
da build apache

Please note that if you upgrade to a custom DA build, the auto-updater will revert DirectAdmin back to the default version after ~30 minutes. This means it would be best to start apache build immediately after updating DA to custom build.

A quick check to see if apache is patched or not is to test for mod_http2 version. Patched apache will have mod_http2 2.0.37, while the default apache will have 2.0.35. A quick test from CLI would be:
strings /usr/sbin/httpd | grep -q -F 2.0.37 && echo patched || echo non-patched

Thank you for your quick assistance Fln. I have updated Apache according to your instructions. I hope this resolves the issue. For now, everything is working correctly.

 

[fln]

Thanks for testing it out @matkra.

How often would you see crashes without this fix? On our low-traffic test servers we can not reliably trigger this bug and we do not want to push it out blindly. If you would confirm this patch eliminates the http/2 related crashes we would push it out as a hot-fix for everyone.