MySQL crash every day at same time

[Artis]

Hi to all,

I have problem with mysql. Every day in same time I have problems with my database.

Configuration on server:
custombuild 1.2
apache 2.2
mysql 5.6
php 5.3

log from /var/lib/mysql/hostname.err

2013-05-22 09:04:22 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2013-05-22 09:04:22 23857 [Note] Plugin 'FEDERATED' is disabled.
2013-05-22 09:04:22 23857 [Note] InnoDB: The InnoDB memory heap is disabled
2013-05-22 09:04:22 23857 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2013-05-22 09:04:22 23857 [Note] InnoDB: Compressed tables use zlib 1.2.3
2013-05-22 09:04:22 23857 [Note] InnoDB: Using Linux native AIO
2013-05-22 09:04:22 23857 [Note] InnoDB: Not using CPU crc32 instructions
2013-05-22 09:04:22 23857 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2013-05-22 09:04:22 23857 [Note] InnoDB: Completed initialization of buffer pool
2013-05-22 09:04:22 23857 [Note] InnoDB: Highest supported file format is Barracuda.
2013-05-22 09:04:22 23857 [Note] InnoDB: The log sequence numbers 855692753 and 855692753 in ibdata files do not match the log sequence number 2466813149 in the ib_logfiles!
2013-05-22 09:04:22 23857 [Note] InnoDB: Database was not shutdown normally!
2013-05-22 09:04:22 23857 [Note] InnoDB: Starting crash recovery.
2013-05-22 09:04:22 23857 [Note] InnoDB: Reading tablespace information from the .ibd files...
2013-05-22 09:04:22 23857 [Note] InnoDB: Restoring possible half-written data pages 
2013-05-22 09:04:22 23857 [Note] InnoDB: from the doublewrite buffer...
2013-05-22 09:04:23 23857 [Note] InnoDB: 128 rollback segment(s) are active.
2013-05-22 09:04:23 23857 [Note] InnoDB: Waiting for purge to start
2013-05-22 09:04:23 23857 [Note] InnoDB: 5.6.11 started; log sequence number 2466813149
2013-05-22 09:04:23 23857 [Note] Server hostname (bind-address): '*'; port: 3306
2013-05-22 09:04:23 23857 [Note] IPv6 is available.
2013-05-22 09:04:23 23857 [Note]   - '::' resolves to '::';
2013-05-22 09:04:23 23857 [Note] Server socket created on IP: '::'.
2013-05-22 09:04:23 23857 [ERROR] /usr/sbin/mysqld: Error writing file '/var/lib/mysql/hostname.pid' (Errcode: 28 - No space left on device)
2013-05-22 09:04:23 23857 [ERROR] Can't start server: can't create PID file: No space left on device

I see that error is with no space, but I have a lot of memory (server has 4gb ram) and space on hdd is about 70% free

Any hint/advice ??

 

[smtalk]

Do you have a separate /var partition on the system? Are daily backups set in DA?

 

[zEitEr]

I see that error is with no space, but I have a lot of memory (server has 4gb ram) and space on hdd is about 70% free

Any hint/advice ??

Inodes?

df -i

df -h

 

[Artis]

Yes I have enabled DA backup options for users (I'm creating every day backup)

[root@ data]# mount
/dev/sda2 on / type ext3 (rw,usrquota,grpquota)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)

Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/sda2            15138816  349447 14789369    3% /
tmpfs                 474206       1  474205    1% /dev/shm
[root@zagreb data]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2             228G   77G  139G  36% /
tmpfs                 1.9G     0  1.9G   0% /dev/shm
[root@zagreb data]#