Possible memory leak in ModSecurity + NGINX via msc_new_transaction() — anyone seen/fixed this?

[sumant dhiman]

Hi all,

I’m investigating a memory buildup / possible leak in the ModSecurity + NGINX integration path, and Valgrind consistently points to allocations happening during msc_new_transaction().

From the stack trace, the issue seems to originate in ModSecurity’s internal collection handling, specifically around InMemoryPerProcess and Collections initialization during transaction creation.

Valgrind output

==23905== 8,248 bytes in 2 blocks are indirectly lost in loss record 1,437 of 1,467
==23905== at 0x482BBB5: operator new(unsigned int) (vg_replace_malloc.c:338)
==23905== by 0x9543206: allocate (new_allocator.h:104)
==23905== by 0x9543206: std::_Hashtable<std::string, std:air<std::string const, modsecurity::collection::backend::CollectionData>, std::allocator<std:air<std::string const, modsecurity::collection::backend::CollectionData> >, std::__detail::_Select1st, modsecurity::collection::backend::MyEqual, modsecurity::collection::backend::MyHash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, false> >::_M_allocate_buckets(unsigned int) [clone .isra.81] (hashtable.h:779)
==23905== by 0x954715A: std::_Hashtable<std::string, std:air<std::string const, modsecurity::collection::backend::CollectionData>, std::allocator<std:air<std::string const, modsecurity::collection::backend::CollectionData> >, std::__detail::_Select1st, modsecurity::collection::backend::MyEqual, modsecurity::collection::backend::MyHash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, false> >::_M_rehash_aux(unsigned int, std::integral_constant<bool, false>) (hashtable.h:1741)
==23905== by 0x95472C8: std::_Hashtable<std::string, std:air<std::string const, modsecurity::collection::backend::CollectionData>, std::allocator<std:air<std::string const, modsecurity::collection::backend::CollectionData> >, std::__detail::_Select1st, modsecurity::collection::backend::MyEqual, modsecurity::collection::backend::MyHash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, false> >::_M_rehash(unsigned int, unsigned int const&) (hashtable.h:1680)
==23905== by 0x9544633: rehash (hashtable.h:1663)
==23905== by 0x9544633: reserve (hashtable_policy.h:819)
==23905== by 0x9544633: reserve (unordered_map.h:1363)
==23905== by 0x9544633: modsecurity::collection::backend::InMemoryPerProcess::InMemoryPerProcess(std::string const&) (in_memory-per_process.cc:42)
==23905== by 0x9542502: modsecurity::collection::Collections::Collections(modsecurity::collection::Collection*, modsecurity::collection::Collection*, modsecurity::collection::Collection*, modsecurity::collection::Collection*, modsecurity::collection::Collection*) (collections.cc:50)
==23905== by 0x94E7E75: modsecurity::Transaction::Transaction(modsecurity::ModSecurity*, modsecurity::RulesSet*, void*) (transaction.cc:164)
==23905== by 0x94E895E: msc_new_transaction (transaction.cc:1917)
==23905== by 0xB546B6E: ???
==23905== by 0xB5479E4: ???
==23905== by 0x165E98: ngx_http_core_rewrite_phase (ngx_http_core_module.c:929)
==23905== by 0x160ED0: ngx_http_core_run_phases (ngx_http_core_module.c:875)
==23905== by 0x16E29D: ngx_http_process_request (ngx_http_request.c:2212)
==23905== by 0x16F158: ngx_http_process_request_headers (ngx_http_request.c:1601)
==23905== by 0x16F5C0: ngx_http_process_request_line (ngx_http_request.c:1230)
==23905== by 0x14EE65: ngx_epoll_process_events (ngx_epoll_module.c:901)
==23905== by 0x141C81: ngx_process_events_and_timers (ngx_event.c:248)
==23905== by 0x14C5FF: ngx_worker_process_cycle (ngx_process_cycle.c:721)
==23905== by 0x14AB69: ngx_spawn_process (ngx_process.c:212)
==23905== by 0x14C93A: ngx_start_worker_processes (ngx_process_cycle.c:344)
``

What I’m trying to understand

• Has anyone seen a similar memory leak / growth pattern in ModSecurity v3 + NGINX?
• Is this a known issue around transaction creation or in-memory collections?

Any pointers would be appreciated.

Thanks!