maria/mysql by itself made to keep in RAM as much as possible because it fastest storage. So you can limit max memory consumption by limiting: max user connections, total buffers, monitor slow queries, make shorter timeouts - but all this will cause more often requests to disks and total mysql...