VPS went down, out of memory.

[BBM]

My VPS just went down today due to an out of memory situation.
So happens yesterday I noticed free mem appearing kinda low, but didn't think much of it since I had just updated Exim.

Trying to find out what went wrong, I couldn't connect with DA or ssh to it either, so through the console terminal I noticed errors on the screen (attachment).
Also noticed CPU usage seemed to be at 100% but disk IO and network traffic was at 0.

Only thing to do for me was to reset/reboot the server and hope nothing had been fubarred.

I would like to know how this could occur and also why this could bring the entire server down. Also would be nice to prevent this from happening again.
Could there be a service with a memory leak?
Any tips on where to start looking?


(Some pics added of the server's specs and stats.)

 

[Arieh]

A server may get inaccessible when it runs out of memory. It starts using swap memory which is very slow. Usually when a sytem runs into trouble, DA notices the high load and makes a ticket of it. It there also shows a dump of all running processes, you may get a clue there.

Also it's easiest if you catch the high load in the act, then you can simply look what's causing it. Personally I use htop (install: centos, debian has it in apt-get by default) to easily find out what's going on.

Other then that, you can look into /var/log/ - exim, httpd etc.

There are other commands to get system stats as well or to take action on it, like http://sysfence.sourceforge.net/

 

[BBM]

After the reboot I started checking all the logs available in DA, but nothing really stood out to me as being out of whack.
I got notice the server (http & pop) went down at 17:30 and I can see logs going upto 17:29.

I've never had an out of memory issue until now and no extra clients or activities have been added recently.
I find it interesting this happens shortly after I updated the vps to the lastest software versions. Makes me think some things might not be working 100% correctly. (There are still a number of Segmentation fault-errors happening on the server, I mostlikely need to look more closely in to them)