* tired to post this in the apache section but kept getting 500 internal server errors when trying to post thread*
- So, this has been a long time coming.. We have had this problem across 2 different servers and multiple upgrades trying to fix the problem for this site. For some reason a few times during the day our httpd memory usage spikes to server crushing levels until web users experience a database connection error trying to load the site.
We started off on a 4gb ram dedicated.. then upgraded to 8Gb, which did help the problem for a day or two.. but then the memory leak quickly caught up and the server wasnt enough..
.. a few times during the day the pid would shoot to 200+ and our server would run httpd 6-7+Gb(users would experience database connection errors).. then the pid's would die off and the memory load would go back to around 2Gb.. where it seems to sit most of the time for 700-900 web users on site.
Then 2 days ago we switched to a server with a much newer processor and 16gb or ram and we are still having issues. A little bit different now.. not sure if its because the hyper threading or not.. but now under service monitor httpd is running at about 2Gb usage..
httpd (pid 13645 14510 15135 15292 15314 15419 15495 15514 15517 15529 15660 15691 15750 15753 15780 15801 15824 15825 15827 15832 15835 16173 16183 16194 16195 16196 16203 16205 16210 16223 16224 16225 16226 16227 16228 16229 16231 16232 16235 16237 16271 16359 16360 16369 16370 16398 16410 16411 16475 16506 16510 16523 16524 16525 16526 16542 16543 16544 16561 16562 16564 16565 16566 16567 16568 16569 16570 16572 16573 16574 16575 16576 16585 16586 16589 16591 16592 16593 16594 16595 16596 16597 16598 16600 16601 16603 16604 16605 16606 16607 16611 16612 16613 16614 16615 29034 )
3.39GB upon checking while writing this..
but under the process monitor,
top - 02:11:29 up 2 days, 11:04, 0 users, load average: 0.85, 0.70, 0.73
Tasks: 340 total, 3 running, 337 sleeping, 0 stopped, 0 zombie
Cpu(s): 10.2%us, 1.6%sy, 0.0%ni, 88.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 16032276k total, 12708236k used, 3324040k free, 227860k buffers
Swap: 8191992k total, 0k used, 8191992k free, 9071868k cached
we are using 12gb of ram? and only have 3 free?
When we installed/imported direct admin to the new server.. the site was.. FAST... it ran with a ton of free memory, and now 2 days later its slow, laggy and obviously hurting for ram again..
under the process monitor we have:
17173 apache 20 0 229m 37m 3956 S 33.3 0.2 0:00.40 /usr/sbin/httpd -k start -DSSL
15495 apache 20 0 233m 41m 3984 S 27.4 0.3 0:05.26 /usr/sbin/httpd -k start -DSSL
15827 apache 20 0 230m 37m 3988 S 25.5 0.2 0:04.46 /usr/sbin/httpd -k start -DSSL
16561 apache 20 0 230m 37m 3980 S 25.5 0.2 0:01.68 /usr/sbin/httpd -k start -DSSL
17150 apache 20 0 226m 32m 3596 R 17.6 0.2 0:00.09 /usr/sbin/httpd -k start -DSSL
9356 mysql 20 0 2910m 66m 5280 S 3.9 0.4 89:41.62 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --log-error=/var/lib/mysql/AO14.worldstream.nl.err --pid-file=/var/lib/mysql/AO14.worldstream.nl.pid
16603 apache 20 0 230m 37m 3988 S 3.9 0.2 0:02.21 /usr/sbin/httpd -k start -DSSL
15780 apache 20 0 230m 37m 3980 S 2.0 0.2 0:01.91 /usr/sbin/httpd -k start -DSSL
17140 apache 20 0 217m 24m 3876 S 2.0 0.2 0:00.18 /usr/sbin/httpd -k start -DSSL
17142 apache 20 0 230m 37m 3952 S 2.0 0.2 0:00.41 /usr/sbin/httpd -k start -DSSL
17149 apache 20 0 230m 37m 3980 S 2.0 0.2 0:00.52 /usr/sbin/httpd -k start -DSSL
Im not super server techy, but i've researched enough to see this: /usr/sbin/httpd -k start -DSSL has something to do with SSL and restarting apache?
Quite frankly i was going to upgrade again to 32Gb of ram.. but, i feel at this point the site is just going to freight train until it eats 32gb instead of 16..
Any thoughts and help on this would be greatly appreciated.. even If I could get some pointers in the right direction.
I dont think its a ddos.. we've had less than 3 brute force attacks on this new server.. (I thought that may be the case on the old server because we got hundreds a day)
Again, any help would greatly be appreciated.
Thanks for your time!
* the site is a wordpress site.. that runs mainly through a plugin.. that does crons every 3 hours... Perhaps the crons are just never closing..?
Server info: Intel(R) Xeon(R) CPU E3-1275 V2 @ 3.50GHz 16Gb ram.. cent os 6 64bit
Let me know if you need any more info, I'll be watching this thread like a hawk.
- So, this has been a long time coming.. We have had this problem across 2 different servers and multiple upgrades trying to fix the problem for this site. For some reason a few times during the day our httpd memory usage spikes to server crushing levels until web users experience a database connection error trying to load the site.
We started off on a 4gb ram dedicated.. then upgraded to 8Gb, which did help the problem for a day or two.. but then the memory leak quickly caught up and the server wasnt enough..
.. a few times during the day the pid would shoot to 200+ and our server would run httpd 6-7+Gb(users would experience database connection errors).. then the pid's would die off and the memory load would go back to around 2Gb.. where it seems to sit most of the time for 700-900 web users on site.
Then 2 days ago we switched to a server with a much newer processor and 16gb or ram and we are still having issues. A little bit different now.. not sure if its because the hyper threading or not.. but now under service monitor httpd is running at about 2Gb usage..
httpd (pid 13645 14510 15135 15292 15314 15419 15495 15514 15517 15529 15660 15691 15750 15753 15780 15801 15824 15825 15827 15832 15835 16173 16183 16194 16195 16196 16203 16205 16210 16223 16224 16225 16226 16227 16228 16229 16231 16232 16235 16237 16271 16359 16360 16369 16370 16398 16410 16411 16475 16506 16510 16523 16524 16525 16526 16542 16543 16544 16561 16562 16564 16565 16566 16567 16568 16569 16570 16572 16573 16574 16575 16576 16585 16586 16589 16591 16592 16593 16594 16595 16596 16597 16598 16600 16601 16603 16604 16605 16606 16607 16611 16612 16613 16614 16615 29034 )
3.39GB upon checking while writing this..
but under the process monitor,
top - 02:11:29 up 2 days, 11:04, 0 users, load average: 0.85, 0.70, 0.73
Tasks: 340 total, 3 running, 337 sleeping, 0 stopped, 0 zombie
Cpu(s): 10.2%us, 1.6%sy, 0.0%ni, 88.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 16032276k total, 12708236k used, 3324040k free, 227860k buffers
Swap: 8191992k total, 0k used, 8191992k free, 9071868k cached
we are using 12gb of ram? and only have 3 free?
When we installed/imported direct admin to the new server.. the site was.. FAST... it ran with a ton of free memory, and now 2 days later its slow, laggy and obviously hurting for ram again..
under the process monitor we have:
17173 apache 20 0 229m 37m 3956 S 33.3 0.2 0:00.40 /usr/sbin/httpd -k start -DSSL
15495 apache 20 0 233m 41m 3984 S 27.4 0.3 0:05.26 /usr/sbin/httpd -k start -DSSL
15827 apache 20 0 230m 37m 3988 S 25.5 0.2 0:04.46 /usr/sbin/httpd -k start -DSSL
16561 apache 20 0 230m 37m 3980 S 25.5 0.2 0:01.68 /usr/sbin/httpd -k start -DSSL
17150 apache 20 0 226m 32m 3596 R 17.6 0.2 0:00.09 /usr/sbin/httpd -k start -DSSL
9356 mysql 20 0 2910m 66m 5280 S 3.9 0.4 89:41.62 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --log-error=/var/lib/mysql/AO14.worldstream.nl.err --pid-file=/var/lib/mysql/AO14.worldstream.nl.pid
16603 apache 20 0 230m 37m 3988 S 3.9 0.2 0:02.21 /usr/sbin/httpd -k start -DSSL
15780 apache 20 0 230m 37m 3980 S 2.0 0.2 0:01.91 /usr/sbin/httpd -k start -DSSL
17140 apache 20 0 217m 24m 3876 S 2.0 0.2 0:00.18 /usr/sbin/httpd -k start -DSSL
17142 apache 20 0 230m 37m 3952 S 2.0 0.2 0:00.41 /usr/sbin/httpd -k start -DSSL
17149 apache 20 0 230m 37m 3980 S 2.0 0.2 0:00.52 /usr/sbin/httpd -k start -DSSL
Im not super server techy, but i've researched enough to see this: /usr/sbin/httpd -k start -DSSL has something to do with SSL and restarting apache?
Quite frankly i was going to upgrade again to 32Gb of ram.. but, i feel at this point the site is just going to freight train until it eats 32gb instead of 16..
Any thoughts and help on this would be greatly appreciated.. even If I could get some pointers in the right direction.
I dont think its a ddos.. we've had less than 3 brute force attacks on this new server.. (I thought that may be the case on the old server because we got hundreds a day)
Again, any help would greatly be appreciated.
Thanks for your time!
* the site is a wordpress site.. that runs mainly through a plugin.. that does crons every 3 hours... Perhaps the crons are just never closing..?
Server info: Intel(R) Xeon(R) CPU E3-1275 V2 @ 3.50GHz 16Gb ram.. cent os 6 64bit
Let me know if you need any more info, I'll be watching this thread like a hawk.