httpd' on server suddenly stop

John, if its a semaphore problem, why isn't Apache releasing the semaphores on shutdown/etc, or why doesn't the system release the semaphore if Apache is no longer running?

Surely you do not need to get rid of the semaphores every time this happens? Is it a known CentOS issue?
 
well im using centos from 2month with directadmin now and never had this problem (120+ domains on it).

Before centos i was using debian and never had those problem there too.

So, im out of ideas...
 
Hi,
first i have typed :
Code: 
ipcs | grep apache | awk '{print $2}' > sem.txt
for i in `cat sem.txt`; do { ipcrm -s $i; }; done;

then i have typed "cat sem.txt" the result:
Code: 
26247969
26280738
26313507
26346276
26379045
26411814
26444583
26477352
26510121
26542890
26575659
26608428
26641197
26673966
26706735
26739504
26772273
26805042
26837811
26870580
26903349
26936118
26968887
27001656
27034425
27067194
27099963
27132732
27165501
27198270
27231039
27263808
27296577
27329346
27362115
27394884
27427653
27460422
27493191
27525960
27558729
27591498
27624267
27657036
27689805
27722574
27755343
27788112
27820881
27853650
27886419
27919188
27951957
27984726
28017495
28050264
28083033
28115802
28148571
28181340
28214109
28246878
28279647
28312416
28345185
28377954
28410723
28443492
28476261
28509030
28541799
28574568
28607337
28640106
28672875
28705644
28738413
28771182
28803951
28836720
28869489
28902258
28935027
28967796
29000565
29033334
29066103
29098872
29131641
29164410
29197179
29229948
29262717
29295486
29328255
29361024
29393793
29426562
29459331
29492100
29524869
29557638
29590407
29623176
29655945
29688714
29721483
29754252
29787021
29819790
29852559
29885328
29918097
29950866
29983635
30016404
30049173
30081942
30114711
30147480
30180249
30213018
30245787
30278556
30311325
30344094
30376863
30409632
30442401
30475170
30507939
30540708
30573477
30606246
30639015
30671784
30704553
30737322
30770091
30802860
30835629
30868398
30901167
30933936
30966705
30999474
31032243
31065012
31097781
31130550
31163319
31196088
31228857
31261626
31294395
31327164
31359933
31392702
31425471
31458240
31491009
31523778
31556547
31589316
31622085
31654854
31687623
31720392
31753161
31785930
31818699
31851468
31884237
31917006
31949775
31982544
32015313
32048082
32080851
32113620
32146389
32179158
32211927
32244696
32277465
32310234
32343003
32375772
32408541
32441310
32474079
32506848
32539617
32572386
32605155
32637924
32670693
32703462
32736231
32769000
32801769
32834538
32867307
32900076
32932845
32965614
32998383
33031152
33063921
33096690
33129459
33162228
33194997
33227766
33260535
33293304
33326073
33358842
33391611
33424380
33457149
33489918
33522687
[root@server ~]#
 
i can know just if i do restart to httpd
4 users ask me to leave the server because this problem
3 days server stop all time
i afraid to do restart now
 
if you wanna solve the problem you have to risk and restart server, cant expect that we online always and fully avaible when server stop working from his own way :)

So better risk an httpd restart that fail and then a os restart.. or.. wait till is night and do the test in the night, but, depending on where we live not all should be online ready to reply..

regards
 
So, you can wait until it gets stuck again late at night, when you sleep, and you'll get a longer downtime. Try it.

Let me do some conclusion, you've been told from several users mostly one thing (re-red the thread): it's a disc space issue or semaphores, file descriptors. But the problem is still here. You can spend more time, days and weeks to find solutions, or spend some dollars to get your server working. Please, hire somebody (not me), and don't wait until your customers will leave you.
 
SeLLeRoNe said:
if you wanna solve the problem you have to risk and restart server, cant expect that we online always and fully avaible when server stop working from his own way :)

So better risk an httpd restart that fail and then a os restart.. or.. wait till is night and do the test in the night, but, depending on where we live not all should be online ready to reply..

regards
Click to expand...

now i will test it man thank you
 
zEitEr said:
So, you can wait until it gets stuck again late at night, when you sleep, and you'll get a longer downtime. Try it.

Let me do some conclusion, you've been told from several users mostly one thing (re-red the thread): it's a disc space issue or semaphores, file descriptors. But the problem is still here. You can spend more time, days and weeks to find solutions, or spend some dollars to get your server working. Please, hire somebody (not me), and don't wait until your customers will leave you.
Click to expand...

You're right in every word, and i told you that what all guys do in this forum is wondrfull !
I hired someone for 6 years
He was doing everything I was focused in building sites
Now he went out of state
some guy ask me for alot of money to check the server only!
if i need to hire somebody i will do!!
but what happen If tomorrow came another problem?
 
now iwill go to do restart httpd
if that not work ,before reboot what you want me to put here? (i mean logs)
thank you
 
asishlla said:
if i need to hire somebody i will do!!
but what happen If tomorrow came another problem?
Click to expand...

Ok, that's your choice, and I respect it, even though I do not understand it. I can hardly remember even a name of a human or company that would like to tolerate their sites' endless downtimes. Probably your customers are of another sort, but not mine.

That's your business and your money, just a piece of advice, don't waste them. Please keep doing what you are good at, and leave others their work. A good specialist can not be cheap. Ask John or Jeff for a paid help, if they agree, of course.

That's all. I'm not going to persuade you any more.
 
Ok, now i do
Code: 
ipcs | grep apache | awk '{print $2}' > sem.txt
for i in `cat sem.txt`; do { ipcrm -s $i; }; done;

then:
Code: 
ipcs

then :
Code: 
service httpd restart

Code: 
[root@server ~]#
[root@server ~]# service httpd restart
Stopping httpd:                                            [  OK  ]
Starting httpd: [Tue Mar 01 11:49:23 2011] [warn] module clamav_module is alread             y loaded, skipping

[root@server ~]#

the result of
Code: 
ipcs


Code: 
[root@server ~]# ipcs

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status
0x7401814d 393216     root      600        4          0
0x74018111 851969     root      600        4          0
0x740180e6 1507330    root      600        4          0
0x00000000 1867779    root      644        224        1
0x740180e5 1474564    root      600        4          0
0x00000000 1900549    root      644        93712      1
0x00000000 1933318    root      644        6277240    1
0x00000000 1966087    root      644        4749568    1
0x00000000 1998856    root      644        857728     1
0x00000000 2031625    root      644        9584       1
0x00000000 2064394    root      644        4176       1
0x00000000 2097163    root      644        19512      1
0x00000000 2129932    root      644        3552       1
0x00000000 2162701    root      644        86468256   1
0x00000000 2195470    root      644        14806328   1
0x00000000 2228239    root      644        5403656    1
0x00000000 2261008    root      644        369936     1
0x00000000 2293777    root      644        9776       1
0x00000000 2326546    root      644        1371832    1
0x00000000 2359315    root      644        155120     1
0x00000000 2392084    root      644        80         1
0x00000000 2424853    root      644        64         1
0x00000000 2457622    root      644        64         1
0x00000000 2490391    root      644        5576       1
0x00000000 2523160    root      644        248        1
0x00000000 2555929    root      644        64         1
0x00000000 2588698    root      644        513680     1
0x00000000 2621467    root      644        80         1
0x00000000 2654236    root      644        227944     1
0x00000000 2687005    root      644        48         1
0x00000000 6619166    root      600        16777216   4          dest
0x00000000 6651935    root      600        524288     4          dest

------ Semaphore Arrays --------
key        semid      owner      perms      nsems
0x00000000 67108864   apache    600        1
0x00000000 67141633   apache    600        1
0x00000000 67174402   apache    600        1
0x00000000 67207171   apache    600        1
0x00000000 67239940   apache    600        1
0x00000000 67272709   apache    600        1
0x00000000 67305478   apache    600        1
0x00000000 67338247   apache    600        1
0x00000000 67371016   apache    600        1
0x00000000 67403785   apache    600        1

------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages

[root@server ~]#

that mean its work?
 
now i typed "ipcs" again the result
Code: 
0x00000000 70484071   apache    600        1
0x00000000 70516840   apache    600        1
0x00000000 70549609   apache    600        1
0x00000000 70582378   apache    600        1
0x00000000 70615147   apache    600        1
0x00000000 70647916   apache    600        1
0x00000000 70680685   apache    600        1
0x00000000 70713454   apache    600        1
0x00000000 70746223   apache    600        1
0x00000000 70778992   apache    600        1
0x00000000 70811761   apache    600        1
0x00000000 70877298   apache    600        1
0x00000000 70910067   apache    600        1
0x00000000 70942836   apache    600        1
0x00000000 70975605   apache    600        1
0x00000000 71008374   apache    600        1
0x00000000 71041143   apache    600        1
0x00000000 71073912   apache    600        1
0x00000000 71106681   apache    600        1
0x00000000 71139450   apache    600        1
0x00000000 71172219   apache    600        1
0x00000000 71204988   apache    600        1
0x00000000 71237757   apache    600        1
0x00000000 71270526   apache    600        1
0x00000000 71303295   apache    600        1
0x00000000 71336064   apache    600        1
0x00000000 71368833   apache    600        1
0x00000000 71401602   apache    600        1
0x00000000 71434371   apache    600        1
0x00000000 71467140   apache    600        1
0x00000000 71499909   apache    600        1
0x00000000 71532678   apache    600        1
0x00000000 71565447   apache    600        1
0x00000000 71598216   apache    600        1
0x00000000 71630985   apache    600        1
0x00000000 71663754   apache    600        1
0x00000000 71696523   apache    600        1
0x00000000 71729292   apache    600        1
0x00000000 71762061   apache    600        1
0x00000000 71794830   apache    600        1
0x00000000 71827599   apache    600        1
0x00000000 71860368   apache    600        1
0x00000000 71893137   apache    600        1
0x00000000 71925906   apache    600        1
0x00000000 71958675   apache    600        1
0x00000000 71991444   apache    600        1
0x00000000 72024213   apache    600        1
0x00000000 72056982   apache    600        1
0x00000000 72089751   apache    600        1
0x00000000 72122520   apache    600        1
0x00000000 72155289   apache    600        1
0x00000000 72188058   apache    600        1
0x00000000 72220827   apache    600        1
0x00000000 72253596   apache    600        1
0x00000000 72286365   apache    600        1
0x00000000 72319134   apache    600        1
0x00000000 72351903   apache    600        1
0x00000000 72384672   apache    600        1
0x00000000 72417441   apache    600        1
0x00000000 72450210   apache    600        1
0x00000000 72482979   apache    600        1
0x00000000 72515748   apache    600        1
0x00000000 72548517   apache    600        1
0x00000000 72581286   apache    600        1
0x00000000 72614055   apache    600        1
0x00000000 72646824   apache    600        1
0x00000000 72679593   apache    600        1
0x00000000 72712362   apache    600        1
0x00000000 72745131   apache    600        1
0x00000000 72777900   apache    600        1
0x00000000 72810669   apache    600        1
0x00000000 72843438   apache    600        1
0x00000000 72876207   apache    600        1
0x00000000 72908976   apache    600        1
0x00000000 72941745   apache    600        1
0x00000000 72974514   apache    600        1
0x00000000 73007283   apache    600        1
0x00000000 73040052   apache    600        1
0x00000000 73072821   apache    600        1
0x00000000 73105590   apache    600        1
0x00000000 73138359   apache    600        1
0x00000000 73171128   apache    600        1
0x00000000 73203897   apache    600        1
0x00000000 73236666   apache    600        1
0x00000000 73269435   apache    600        1
0x00000000 73302204   apache    600        1
0x00000000 73334973   apache    600        1
0x00000000 73367742   apache    600        1
0x00000000 73400511   apache    600        1
0x00000000 73433280   apache    600        1
0x00000000 73466049   apache    600        1
0x00000000 73498818   apache    600        1
0x00000000 73531587   apache    600        1
0x00000000 73564356   apache    600        1
0x00000000 73597125   apache    600        1
0x00000000 73629894   apache    600        1
0x00000000 73662663   apache    600        1
0x00000000 73695432   apache    600        1
0x00000000 73728201   apache    600        1
0x00000000 73760970   apache    600        1
0x00000000 73793739   apache    600        1
0x00000000 73826508   apache    600        1
0x00000000 73859277   apache    600        1
0x00000000 73892046   apache    600        1
0x00000000 73924815   apache    600        1
0x00000000 73957584   apache    600        1
0x00000000 73990353   apache    600        1
0x00000000 74023122   apache    600        1
0x00000000 74055891   apache    600        1
0x00000000 74088660   apache    600        1
0x00000000 74121429   apache    600        1
0x00000000 74154198   apache    600        1
0x00000000 74186967   apache    600        1
0x00000000 74219736   apache    600        1
0x00000000 74252505   apache    600        1
0x00000000 74285274   apache    600        1
0x00000000 74318043   apache    600        1
0x00000000 74350812   apache    600        1
0x00000000 74383581   apache    600        1
0x00000000 74416350   apache    600        1
0x00000000 74449119   apache    600        1
0x00000000 74481888   apache    600        1
0x00000000 74514657   apache    600        1
0x00000000 74547426   apache    600        1
0x00000000 74580195   apache    600        1
0x00000000 74612964   apache    600        1
0x00000000 74645733   apache    600        1
0x00000000 74678502   apache    600        1
0x00000000 74711271   apache    600        1
0x00000000 74744040   apache    600        1
0x00000000 74776809   apache    600        1
0x00000000 74809578   apache    600        1
0x00000000 74842347   apache    600        1
0x00000000 74875116   apache    600        1
0x00000000 74907885   apache    600        1
0x00000000 74940654   apache    600        1
0x00000000 74973423   apache    600        1
0x00000000 75006192   apache    600        1
0x00000000 75038961   apache    600        1
0x00000000 75071730   apache    600        1
0x00000000 75104499   apache    600        1
0x00000000 75137268   apache    600        1
0x00000000 75170037   apache    600        1
0x00000000 75202806   apache    600        1
0x00000000 75235575   apache    600        1
0x00000000 75268344   apache    600        1
0x00000000 75301113   apache    600        1
0x00000000 75333882   apache    600        1
0x00000000 75366651   apache    600        1
0x00000000 75399420   apache    600        1
0x00000000 75432189   apache    600        1
0x00000000 75464958   apache    600        1
0x00000000 75497727   apache    600        1
0x00000000 75530496   apache    600        1
0x00000000 75563265   apache    600        1
0x00000000 75596034   apache    600        1
0x00000000 75628803   apache    600        1
0x00000000 75661572   apache    600        1
0x00000000 75694341   apache    600        1
0x00000000 75727110   apache    600        1
0x00000000 75759879   apache    600        1
0x00000000 75792648   apache    600        1
0x00000000 75825417   apache    600        1
0x00000000 75858186   apache    600        1
0x00000000 75890955   apache    600        1
0x00000000 75923724   apache    600        1
0x00000000 75956493   apache    600        1
0x00000000 75989262   apache    600        1
0x00000000 76022031   apache    600        1
0x00000000 76054800   apache    600        1
0x00000000 76087569   apache    600        1
0x00000000 76120338   apache    600        1
0x00000000 76153107   apache    600        1
0x00000000 76185876   apache    600        1
0x00000000 76218645   apache    600        1
0x00000000 76251414   apache    600        1
0x00000000 76284183   apache    600        1
0x00000000 76316952   apache    600        1
0x00000000 76349721   apache    600        1
0x00000000 76382490   apache    600        1
0x00000000 76415259   apache    600        1
0x00000000 76448028   apache    600        1
0x00000000 76480797   apache    600        1
0x00000000 76513566   apache    600        1
0x00000000 76546335   apache    600        1
0x00000000 76579104   apache    600        1
0x00000000 76611873   apache    600        1
0x00000000 76644642   apache    600        1
0x00000000 76677411   apache    600        1
0x00000000 76710180   apache    600        1
0x00000000 76742949   apache    600        1
0x00000000 76775718   apache    600        1
0x00000000 76808487   apache    600        1
0x00000000 76841256   apache    600        1
0x00000000 76874025   apache    600        1
0x00000000 76906794   apache    600        1
0x00000000 76939563   apache    600        1
0x00000000 76972332   apache    600        1
0x00000000 77005101   apache    600        1
0x00000000 77037870   apache    600        1
0x00000000 77070639   apache    600        1
0x00000000 77103408   apache    600        1
0x00000000 77136177   apache    600        1
0x00000000 77168946   apache    600        1
0x00000000 77201715   apache    600        1
0x00000000 77234484   apache    600        1
0x00000000 77267253   apache    600        1
0x00000000 77300022   apache    600        1
0x00000000 77332791   apache    600        1
0x00000000 77365560   apache    600        1
0x00000000 77398329   apache    600        1
0x00000000 77431098   apache    600        1
0x00000000 77463867   apache    600        1
0x00000000 77496636   apache    600        1
0x00000000 77529405   apache    600        1
0x00000000 77562174   apache    600        1
0x00000000 77594943   apache    600        1
0x00000000 77627712   apache    600        1
0x00000000 77660481   apache    600        1
0x00000000 77693250   apache    600        1
0x00000000 77726019   apache    600        1
0x00000000 77758788   apache    600        1
0x00000000 77791557   apache    600        1
0x00000000 77824326   apache    600        1
0x00000000 77857095   apache    600        1
0x00000000 77889864   apache    600        1
0x00000000 77922633   apache    600        1
0x00000000 77955402   apache    600        1
0x00000000 77988171   apache    600        1
0x00000000 78020940   apache    600        1
0x00000000 78053709   apache    600        1
0x00000000 78086478   apache    600        1
0x00000000 78119247   apache    600        1
0x00000000 78152016   apache    600        1

------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages

[root@server ~]#
 
Code: 
[root@server ~]#
[root@server ~]# service httpd restart
Stopping httpd:                                            [  OK  ]
Starting httpd: [Tue Mar 01 11:49:23 2011] [warn] module clamav_module is alread             y loaded, skipping

[root@server ~]#

I dont see any error here, so seems apache started without problem and just with a warning that you can ignore.. am i wrong?
 
yes SeLLeRoNe it's true
after i do what John said ,now I succeeded restart httpd via root without problem.
My question is: Is this the problem is over?
John said:
You may need to have a cronjob to clear out your semaphores on a regular basis if they're filling up too quickly.
Click to expand...
how can i do that?
thank you
 
Code: 
echo "ipcs | grep apache | awk \'{print $2}\' > /tmp/sem.txt" >> /root/sem.sh
echo "for i in \`cat /tmp/sem.txt\`; do { ipcrm -s $i; }; done;" >> /root/sem.sh
echo "rm /tmp/sem.txt" >> /root/sem.sh
exho "service httpd restart" >> /root/sem.sh
echo "0 * * * * root /root/sem.sh >/dev/null 2>&1" >> /etc/crontab
chmod 700 /root/sem.sh

Do this in shell.

This will setup a script called sem.sh in your root directory and will add a crontab for execute this script once every hour and restart apache.

If anyone got other ideas to do it better please dont be shy :)

Regards
 
The cron is the solution if you want 90% automation.

However, pardon my ignorance, why is there a semaphore issue? Is it a configuration issue, like a maximum of semaphores a box can use, or just an exhausted server thats been pounded with high traffic?

I found this [bergie.iki.fi] - the link to a forum on there is also a interesting read.
 
Hi peter
i did not do what SeLLeRoNe said yet,,i wait for A late hour night to do that.
peter if you think this is configuration issue,,What do you suggest?
or just an exhausted server thats been pounded with high traffic?
Click to expand...
server cost 185£ per month can not hold small 60 sites?
------------------------
- Intel Core2 Quad Q9650
- 8GB Memory
- 2x 500GB 7200rpm Hard Disk
- CentOS 5.x
- JBMC Software DirectAdmin
- 10 TB Bandwidth
------------------------
can you signin MSN for few minutes pls?
Regards
 
is this site talk about that peter?
http://www.notesbit.com/index.php/s...-memory-semaphore-values-for-oracle-database/

Code: 
[root@server ~]# ipcs -ls

------ Semaphore Limits --------
max number of arrays = 1024
max semaphores per array = 250
max semaphores system wide = 256000
max ops per semop call = 32
semaphore max value = 32767

[root@server ~]#

my sysctl.conf :
Code: 
# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled.  See sysctl(8) and
# sysctl.conf(5) for more details.

# Controls IP packet forwarding
net.ipv4.ip_forward = 0

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0

# Controls whether core dumps will append the PID to the core filename
# Useful for debugging multi-threaded applications
kernel.core_uses_pid = 1

# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1

# Controls the maximum size of a message, in bytes
kernel.msgmnb = 65536

# Controls the default maxmimum size of a mesage queue
kernel.msgmax = 65536

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 4294967295

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 268435456

kernel.msgmni = 1024
kernel.sem = 250 256000 32 1024
 
Last edited:
You must log in or register to reply here.
Back
Top