Weird problem with backups not running and than disk full while disk isn't full

Z

Zwerver

Verified User
Joined
Apr 4, 2006
Messages
23
Hi,

It's been a long time that I asked a question here, but now I'm really stuck.

I'm running a Debian 7.2 installation with DA complete up2date. Everytime a users starts a backup; the following happens:

/var/log/directadmin/error.log
Code: 
2014:03:14-07:24:29: Can't connect to ssl!
2014:03:14-07:24:29: -> A failure in the SSL library occurred, usually a protocol error.
2014:03:14-07:24:29: -> error:00000001:lib(0):func(0):reason(1)
2014:03:14-07:24:30: Unable to read ./data/admin/login.hist: Unable to get Lock on file:<br>
open error for ./data/admin/login.hist.lock: Operation not permitted<br>
./data/admin: diradmin:diradmin drwx------, running as diradmin:haldaemon<br>

2014:03:14-07:24:31: Unable to write ./data/admin/login.hist: Unable to get Lock on file:<br>
open error for ./data/admin/login.hist.lock: Operation not permitted<br>
./data/admin: diradmin:diradmin drwx------, running as diradmin:haldaemon<br>
Unable to get Lock on file:<br>
open error for ./data/admin/login.hist.lock: Operation not permitted<br>
./data/admin: diradmin:diradmin drwx------, running as diradmin:haldaemon<br>

[B]<.....SNIP....>[/B]

2014:03:14-07:27:13: Unable to read ./data/admin/login.hist: Unable to get Lock on file:<br>
open error for ./data/admin/login.hist.lock: Operation not permitted<br>
./data/admin: diradmin:diradmin drwx------, running as diradmin:haldaemon<br>

2014:03:14-07:27:14: Unable to write ./data/admin/login.hist: Unable to get Lock on file:<br>
open error for ./data/admin/login.hist.lock: Operation not permitted<br>
./data/admin: diradmin:diradmin drwx------, running as diradmin:haldaemon<br>
Unable to get Lock on file:<br>
open error for ./data/admin/login.hist.lock: Operation not permitted<br>
./data/admin: diradmin:diradmin drwx------, running as diradmin:haldaemon<br>

After that, no file can be created on the file system any more, because of disk full errors, while df show 72% full disk (e.g. 28% free space). Checking lsof -n | grep deleted shows me that there are a lot of deleted files open, but after a while the system restores itself (probably because the open deleted files are released?).

So, my two main questions:
  • which process is spamming the error log, as it is clearly running under te wrong user and I need to fix that.
  • is there a way to circumvent the full disk errors that you are aware off?

To add to that, the load on the system sky-rockets, but that's only logical as the io goes up wildly.
Thanks in advance for all the help!
 
Last edited:
Hi Jeff,

In the end yes, there was a spammer active on the system and because of the rate limiting for outbound emails, the queue was completely stuffed. These queued emails were eating up all the inodes which caused the system to say: no free disk space. In the end I scripted something to remove the messages one by one based on the sender domain and after 2 days all was cleaned up.

Thanks for asking!
 
And thanks for responding in detail which will help future forum visitors with the same problem, so they'll be able to more-easily troubleshoot.

Jeff
 
You must log in or register to reply here.
Back
Top