MySQL server crashed randomly (but on same time of day)

[Jensemann]

Hi!
Every night the server seems to restart itself, but now and then (twice a month), the whole mysql server crashes, and we need to take the server down for a reboot.

Does anyone have any similar experiences?
(Built using custombuild- latest mysql 5 version, allthough previous versions has had the same problem (v5 only))

081119 02:32:57 mysqld started
081119 2:32:57 [Warning] Asked for 196608 thread stack, but got 126976
081119 2:32:58 InnoDB: Started; log sequence number 0 421085
081119 2:32:58 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.0.67-community' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Edition (GPL)
081120 2:33:02 [Note] /usr/sbin/mysqld: Normal shutdown

081120 2:33:03 InnoDB: Starting shutdown...
081120 2:33:03 InnoDB: Shutdown completed; log sequence number 0 421085
081120 2:33:03 [Note] /usr/sbin/mysqld: Shutdown complete

081120 02:33:03 mysqld ended

081120 02:33:20 mysqld started
081120 2:33:20 [Warning] Asked for 196608 thread stack, but got 126976
081120 2:33:21 InnoDB: Started; log sequence number 0 421085
081120 2:33:21 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.0.67-community' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Edition (GPL)
081121 2:32:42 [Note] /usr/sbin/mysqld: Normal shutdown

081121 2:32:42 InnoDB: Starting shutdown...
081121 2:32:43 InnoDB: Shutdown completed; log sequence number 0 421085
081121 2:32:43 [Note] /usr/sbin/mysqld: Shutdown complete

081121 02:32:43 mysqld ended

081121 02:33:00 mysqld started
081121 02:33:00 mysqld started
^G/usr/sbin/mysqld: Error on delete of '/var/lib/mysql/server.lower-test' (Errcode: 2)081121 2:33:00 [Warning] Asked for 196608 thread stack, but got 126976

081121 2:33:00 [Warning] Asked for 196608 thread stack, but got 126976
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
081121 2:33:00 InnoDB: Retrying to lock the first data file
081121 2:33:01 InnoDB: Started; log sequence number 0 421085
081121 2:33:01 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.0.67-community' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Edition (GPL)
081121 2:34:40 InnoDB: Unable to open the first data file
InnoDB: Error in opening ./ibdata1
081121 2:34:40 InnoDB: Operating system error number 11 in a file operation.
InnoDB: Error number 11 means 'Resource temporarily unavailable'.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/operating-system-error-codes.html
InnoDB: Could not open or create data files.
InnoDB: If you tried to add new data files, and it failed here,
InnoDB: you should now edit innodb_data_file_path in my.cnf back
InnoDB: to what it was, and remove the new ibdata files InnoDB created
InnoDB: in this failed attempt. InnoDB only wrote those files full of
InnoDB: zeros, but did not yet use them in any way. But be careful: do not
InnoDB: remove old data files which contain your precious data!
081121 2:34:40 [ERROR] Can't start server: Bind on TCP/IP port: Address already in use
081121 2:34:40 [ERROR] Do you already have another mysqld server running on port: 3306 ?
081121 2:34:40 [ERROR] Aborting

081121 2:34:40 [Note] /usr/sbin/mysqld: Shutdown complete

081121 02:34:40 mysqld ended

081121 02:45:55 mysqld started
081121 2:45:55 [Warning] Asked for 196608 thread stack, but got 126976
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
081121 2:45:55 InnoDB: Retrying to lock the first data file
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.

InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
081121 2:47:35 InnoDB: Unable to open the first data file
InnoDB: Error in opening ./ibdata1
081121 2:47:35 InnoDB: Operating system error number 11 in a file operation.
InnoDB: Error number 11 means 'Resource temporarily unavailable'.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/operating-system-error-codes.html
InnoDB: Could not open or create data files.
InnoDB: If you tried to add new data files, and it failed here,
InnoDB: you should now edit innodb_data_file_path in my.cnf back
InnoDB: to what it was, and remove the new ibdata files InnoDB created
InnoDB: in this failed attempt. InnoDB only wrote those files full of
InnoDB: zeros, but did not yet use them in any way. But be careful: do not
InnoDB: remove old data files which contain your precious data!
081121 2:47:35 [ERROR] Can't start server: Bind on TCP/IP port: Address already in use
081121 2:47:35 [ERROR] Do you already have another mysqld server running on port: 3306 ?
081121 2:47:35 [ERROR] Aborting

081121 2:47:35 [Note] /usr/sbin/mysqld: Shutdown complete

081121 02:47:35 mysqld ended

081121 03:55:09 mysqld started
081121 3:55:09 [Warning] Asked for 196608 thread stack, but got 126976
InnoDB: Unable to lock ./ibdata1, error: 11

 

[scsi]

Are you running the latest mysql version?

 

[Jensemann]

I'm running the latest yes.

 

[alexo]

Unfortunately the same is here

here is output from mysqld.log file

090204 14:04:25 mysqld ended

090204 14:04:28 mysqld started
090204 14:04:28 [Warning] Asked for 196608 thread stack, but got 126976
090204 14:04:28 InnoDB: Started; log sequence number 0 39242358
090204 14:04:28 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.0.67-community-log' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Edition (GPL)
A mysqld process already exists at Wed Feb 4 14:04:41 CST 2009
A mysqld process already exists at Wed Feb 4 14:06:17 CST 2009
A mysqld process already exists at Wed Feb 4 14:07:48 CST 2009
A mysqld process already exists at Wed Feb 4 15:11:04 CST 2009
A mysqld process already exists at Wed Feb 4 15:12:35 CST 2009
A mysqld process already exists at Wed Feb 4 15:14:05 CST 2009
A mysqld process already exists at Wed Feb 4 15:15:35 CST 2009
A mysqld process already exists at Wed Feb 4 15:17:07 CST 2009
A mysqld process already exists at Wed Feb 4 15:18:37 CST 2009

Any advice will be appreciated.
Unfortunately i'm newbie in linux , so don't know what to do ?

Server have 2-3 downtimes/day , even try to install Monit[url], but even monit cann't monitor mysqld

Process 'mysql'
status not monitored
monitoring status not monitored
data collected Wed Feb 4 16:26:11 2009

any advice

P.s. i'm on MySQL 5.0.67,