[bug] Root owned backups are not counted in OpenVZ container

[zEitEr]

Hello,

The box is an OpenVZ container with enabled quotas on /dev/simfs .

We've just found a user who uses 2,3 times of his allowed disk space of 1Gb.
And here is what we've found:

# du -h /home/someuser/ --max-depth=1
4,0K    /home/someuser/.spamassassin
1,7G    /home/someuser/backups
32K     /home/someuser/Maildir
28K     /home/someuser/imap
680M    /home/someuser/domains
2,3G    /home/someuser/

and Directadmin seems to not count files located in /home/someuser/backups owned by root user:

# /usr/local/directadmin/dataskq d800
Debug mode. Level 800

root priv set: uid:0 gid:0 euid:0 egid:0
pidfile written
staring queue
dataskq: command: action=quotatally&value=someuser&type=user
Tally User someuser Begin: mask 20
                email user /home/someuser/Maildir/.INBOX.spam
                email user /home/someuser/Maildir/.INBOX.Trash
                email user /home/someuser/Maildir/.Trash
someuser: Purging spam emails older than 30 days for mimc.org.ru
                email user /home/someuser/imap/mimc.org.ru/admin/Maildir/.INBOX.spam
                email user /home/someuser/imap/mimc.org.ru/admin/Maildir/.INBOX.Trash
                email user /home/someuser/imap/mimc.org.ru/admin/Maildir/.Trash
  /home/someuser/Maildir is 32768 : realtotal: 32768
  /home/someuser/imap is 28672 : realtotal: 61440
  /home/someuser/.spamassassin/spam is 0 : total: 0 realtotal: 61440
  /home/someuser/.spamassassin/user_spam is 0 : total: 0 realtotal: 61440
  /var/spool/virtual/mimc.org.ru is 0 : total: 0 realtotal: 61440
  Majordomo usage for /etc/virtual/mimc.org.ru/majordomo : 4096
  databases for someuser
  Counting found totals for someuser
total bw is 0.000000 from (null) (long long: 0)
bw in meg: 0.000000
meg converted to a string: 0.0000
  spaceused: 755986432.000000 (just quota)
  spaceused: 760455435.000000 (add dbtotal 4469003)
  spaceused: 760459531.000000 (add emltotal 4096) (fyi: realemltotal: 65536)
  spaceused: 760459531.000000 (add apachetotal 0)
  spaceused: 760459531.000000 (add anonftptotal 0)
  spaceused, in meg: 725.230723
  meg to temp: 725.2
  user.usage quota: 725.2  actual: 725.230723 (should match)
  user.uagae email_quota: 65536  actual: 65536  (should match)
ConfigFile::fillFromPost(NULL) : ConnfigFile::filename=(null) : Code last flagged at: getlock(/etc/virtual/mimc.org.ru/autoresponder.conf) : finished
Tally User someuser Complete
done queue

and this is what he has in /home/someuser/backups

# LANG=C ls -la /home/someuser/backups
total 1684288
drwx------ 2 someuser someuser      4096 Jan 21 14:40 .
drwx--x--- 7 someuser access       4096 Dec 12 14:51 ..
-rw-r----- 1 root    root    544344154 Dec 12 14:41 backup-Dec-12-2012-1.tar.gz
-rw-r----- 1 root    root    519610032 Jan 21 09:52 backup-Jan-21-2013-1.tar.gz
-rw-r----- 1 root    root    519617913 Jan 21 10:35 backup-Jan-21-2013-2.tar.gz
-rw-r--r-- 1 someuser someuser         0 Jan 21 14:40 backup-Jan-21-2013-2.tar.gz.tmp
-rw-r----- 1 someuser someuser 139423133 Oct 13  2010 backup-Oct-13-2010-1.tar.gz

And here is what happened (in my understanding):

1. The user having about 70% used space starts creating a backup of his account.
2. As soon as backup is ready Directadmin tries to chown it to someuser, and fails. Thus backup files remains to be owned by root user.
3. As soon as directadmin does not get into account files owned by root in /home/someuser/backups we don't get an alert about the over-usage.

System disk quotas does not work in this case, so the user being unable either to download or delete the files tries to create another backup files, and he faces the same once more time. And in this time he submits a ticket with us.

I'd like this to be fixed. We don't want to allow our users to fill the disk up with their backups owned by root. Neither do the others, I guess.

 

[DirectAdmin Support]

Hmm.. I tested this on our CentOS 6 box (not a VPS).. and it just tested with the following results.

1) strict_backup_permissions=0 allowed the tar.gz to be created, just put him overusage but ownership was correct (set to the user)

2) strict_backup_permissions=1, the tar.gz was created, but got chopped off and tar threw an error

Error Compressing the backup file backup-Jan-21-2013-1.tar.gz : 
gzip: stdout: Disk quota exceeded
/bin/tar: /home/testuser/backups/backup-Jan-21-2013-1.tar.gz: Wrote only 6144 of 10240 bytes
/bin/tar: Error is not recoverable: exiting now

Looking at the DA code, DA is running as root after each case, so shouldn't have any issues changing the chmod value..

Are you able to set the chmod value manually as root, to that of the user while he's over quota?

It may just be a difference in behavior between the different quota systems... where in the VPS, over quota means that a file is not allowed to be changed, even by root... so if we did change something, the only real solution for the code would be to delete the tar.gz.. or send a notice to the Admins.

I have a feeling in your case, the directadmin.conf option:

[url=http://www.directadmin.com/features.php?id=1061]strict_backup_permissions=1[/url]

would resolve it, since the tar would run as the User.. so it wouldn't be set to root in the first place.

This may also be useful:

Admin Level -> Admin Settings -> Allow Users with less than 60% Disk Usage to create Backups. (0 = off, 100 = always on)

try setting it to a lower value that whatever you currently have, so that DA will prevent them from creating a backup in the first place.

John

 

[zEitEr]

OK, I've just tested the same in another VPS hosted on a different OpenVZ hardware node. And here I should say the option works fine and quotas are fine as well. I'd guess it's somehow related to system software (quota package and /proc/mounts) and has nothing to do with Directadmin at all.

The option

strict_backup_permissions=1

fixed the issue here:

gzip: stdout: Disk quota exceeded
/bin/tar: /home/anotheruser/backups/backup/home.tar.gz: Wrote only 4096 of 10240 bytes
/bin/tar: Error is not recoverable: exiting now

Error Compressing the backup file backup-Jan-22-2013-1.tar.gz :
gzip: stdout: Disk quota exceeded
/bin/tar: /home/anotheruser/backups/backup-Jan-22-2013-1.tar.gz: Cannot write: Broken pipe
/bin/tar: Error is not recoverable: exiting now

Thank you, John.


But still important to consider the following thing:

chown: changing ownership of `/home/anotheruser/domains/anotherdomain.com/public_html/big-file.gzip': Disk quota exceeded

 

[nobaloney]

John, please look at the last part of zEitEr's post directly above, specifically the message about disk quota being exceeded.

This is what happens when root changes the ownership of a file to a user and the user is then over quota. This can become a serious problem because as long as disk quota is exceeded for any reason email cannot be delivered, databases cannot be written to, and other issues could occur.

Can you figure out a way around this?

Jeff

 

[DirectAdmin Support]

Hello,

This option will run tar as the User:

[COLOR=#333333]strict_backup_permissions=1

For most cases, this will work fine. Quotas cannot be exceeded with this method, but they can be met.
This would be a per-Admin decision. Sometimes they want the Admin/Reseller backup created, regardless of size.

John[/COLOR]

 

[zEitEr]

John, please look at the last part of zEitEr's post directly above, specifically the message about disk quota being exceeded.

Jeff,

That's specially actual for OpenVZ container with simfs, the things might differ on etx3/ext4 FS. I was successful when chowning files on etx3 which resulted in exceeding disk quotas of a particular user.

I want just say (of course it might differ, depending on the settings and configuration), but in our particular cases (we ran several boxes with OpenVZ, and the situation is the same on them), we can not exceed disk quotas on simfs, and that's why chown fails with that error.

 

[nobaloney]

Yes, on some file systems you can go over quota by chowning a file, but still, once you've done that if you're overlimit you can't receive email, etc., for that user.

It's something to consider.

Jeff