How are disk quotas computed

D

donco

Verified User
Joined
Feb 5, 2004
Messages
45
The following investigation has caused us to wonder the following:
  • How does DirectAdmin tally customer disk usage quotas?
  • Additionally, how does DirectAdmin tally the email disk usage?
  • Should customer IMAP directories be owned by root??

We have a customer on a server, which has a du that looks like this:
Code: 
[root@server:~]# du -sm /home/customer /home/mysql/customer_db \
/var/spool/{mail/customer,virtual/customer_domains}
201     /home/customer
34      /home/mysql/customer_db
1       /var/spool/mail/customer
0       /var/spool/virtual/customer_domain_1
8       /var/spool/virtual/customer_domain_2

This would appear that all disk usage tallies up to ~247MB. In the DirectAdmin web interface, the usage for this customer shows:
Code: 
Total Disk Usage (meg)	247.6
  Email Disk Usage	93.2 Mb	
  Database Disk Usage	32.1 Mb

Which makes sense on the surface. However, neither /home/customer/domains/customer_domain/stats or /home/customer/domains/customer_domain/logs are customer owned (root owned), so they would need to be removed from our estimate, I would think:
Code: 
[root@server:~]# du -sm /home/customer/domains/customer_domain/{stats,logs}
3       /home/customer/domains/customer_domain/stats
17      /home/customer/domains/customer_domain/logs

Further, there are root owned IMAP files which attribute a massive chunk of the du as well:
Code: 
[root@server:/home/customer/imap/customer_domain/user/mail]# ll
total 86260
-rw-rw----    1 customer mail          515 Mar 17  2004 Drafts
-rw-------    1 root     root          510 Mar 24  2004 INBOX.Drafts
-rw-------    1 root     root      1352035 Jun 13 09:18 INBOX.Sent
-rw-------    1 root     root     59751314 Jun 23 09:42 INBOX.Trash
-rw-rw----    1 customer mail        27653 Mar 17  2004 Sent
-rw-rw----    1 customer mail     27080691 Mar 17  2004 Trash

Using a find & tally command I wrote, I was able to discover that /home/customer only has ~116MB of actual customer owned files. See handy dandy oneliner:
Code: 
t=0; for i in `find /home/customer -uid customer_uid -exec ls -ld {} \;|awk '{print $5}'`; \
do t=$[$t+i]; done; echo $t
(Note that the customer doesn't have very many files, so this works)

Naturally, one could also restrict find to only files, which would remove all the 4k directory counts as well, which would further lower the tally.

As a trouble shooting step, we tried setting the IMAP directories to customer ownership & re ran the tally, but that only skewed the disk usage even higher! :eek:

It is also worth mentioning that the email tally appears to be correct with or without the ownership of the IMAP files set (doesn't change, and appears to roughly match a du of /home/customer/imap, /home/customer/mail, /var/spool/mail/customer & /var/spool/virtual/customer/domains).

So, my questions, again, are:
  • How does DirectAdmin tally customer disk usage quotas?
  • Additionally, how does DirectAdmin tally the email disk usage?
  • Should customer IMAP directories be owned by root??
 
Last edited:
Hello,

The disk usage calculation relies on the system quotas for a large chunk of the info, but jumps to manual calculation of files for anything not owned by the user, or files owned by the user on a different partition (eg, /var).

1)
Code: 
/usr/sbin/repquota /home > /home/tmp/quota-dump
is run to get a one time list of all the quotas for the users (where /home is what your quota_partition is set to in your directadmin.conf). You can check to see what each user is getting by typing "quota -v username". If that's higher than you think it should be, see http://help.directadmin.com/item.php?id=39 If that isn't working properly, then see http://help.directadmin.com/item.php?id=42

2) DA then figures out if it should count the email in /var or not. If /var and /home are on the same partition (eg you only have / ), then the system quotas in 1) would have taken care of it, so it would return "no". If they are on seperate partitions, then it would return yes. If you've got a strange setup, you can override it, see: http://www.directadmin.com/features.php?id=488. The email counting would then count the /var/spool/username and /var/spool/domain.com/* for all his domains. While going through this I noticed the /home/user/imap directory fell under that if statment, which it didn't belong, so it will be changed for the next release (might be the problem).

3) Then DA goes around and finds all of the non-user owned files to count up and include. Things like the database usage, all files under /home/username that are owned by apache, the /home/username/imap files (which has just be taken out of the if statement from 2). I don't believe the /home/user/domains/domain.com/stat and logs directories are included...that might be up for changing.

There are quite a few disk usage calculation functions, so if I've missed anything here, let me know and I'll check and see if I missed adding it to this thread, or if it simply hasn't been added.

The imap directories are owned by root because that's how wu-imapd seems to do it. We've adjusted the best we can to work with it. (If/when we ever switch to maildir, that will be addressed)

John
 
John,

What happens for systems owned by those of us who don't use system quotas because we don't want to ever be in a situation where a client can't upload, even if he's gone over his quota?

I don't use system quotas, but it's imortant to me that everything still gets counted. That way we tell our clients when they go over.

Is this a problem for DA's measuring of quotas?

Thanks.

Jeff
 
Hi Jeff,

If quotas are not enabled in the kernel, things can still work correctly. In the /etc/cron.d/directadmin_cron, there is a command to do a quotacheck on the system. If quotas are turned off in the kernel, then this command is essential to keep things updated (should probably set it to run daily).

When quotas are off, but counted using the quota check, DA will show the correct usage, and users will be able to upload more than there limit (because the system isn't enforcing it)

John
 
I'm presuming you mean this line:
Code: 
5 0 * * * root /sbin/quotaoff -a; /sbin/quotacheck -caug; /sbin/quotaon -a;
Well, this explains why I get this email every night:
quotacheck: Cannot remount filesystem mounted on / read-only so counted values might not be right.
Please stop all programs writing to filesystem or use -m flag to force checking.
Click to expand...
I presume everything then is working as it should?

Thanks.

Jeff
 
Hi Jeff,

Looks like a fairly old cron file. Change:

quotacheck -caug

to:

Redhat:
quotacheck -augm

FreeBSD:
quotacheck -aug

John
 
For what it's worth, I'm not sure it's ethical to charge for the apache logs unless there's an option for the customers to have them not kept, and/or that the apache logs be owned by the customer (seems a pretty quick/easy chown).

The reason is, the apache logs can get quite large, and the customer has no way to remove them beyond, of course, a support request to us (and we like to keep our support hit down ;) ).

Beyond that, I appreciate your thorough and insightful answers. Now that I've had this confirmed (I was pretty sure that's what was going on!), I'm off to see about verifying a customer's disk usage...

Regards,
-the donco
 
FAILURE

I've now computed the new usage based on the insights provided above, and I'm still coming up short. The following will illustrate how I computed the allocated data to the block size, included blocks used by directories, and tallied it all up to what DirectAdmin is reporting. My tally needs to match, but it's coming up short & our customer is wanting proof that they are indeed using the disk reported.

First, the following command will display block counts via the ls command:
Code: 
find -user customer -exec ls -lsd --block-size=4096 {} \;|awk '{print $1}'
(BTW: How often does one get to "ls -lsd" :o )
Note, 4KB block size determined by Linux default filesystem block size + verification via the tune2fs command. Depending on where I am, I might run the same find command with "-user root", or no "-user" flag at all, as appropriate.

Since the file counts are generally less than the bash expansion limit on Linux, I just wrap the whole thing in a for loop (otherwise, I would output to a file and iterate over the file with some linecount based while loop w/head&tail wazza):
Code: 
t="0"; for i in `find commands`; do t="$[$t+$i]"; done; echo "$t"
With that primer, here are my findings:

First, DA reports quota = 248.3MB

customer owned files in /home/customer:
31085 4KB blocks

root owned files in /home/customer/imap
16031 4KB blocks

customer user in /var/spool/mail & customer domains in /var/spool/virtual:
1350 4KB blocks

customer databases in /home/mysql:
8729 4KB blocks

total blocks calculated = 57195 4KB blocks
total bytes usage = 234270720
total megabytes usage = 223.4MB

Obviously this is quite a bit shy of the mark.

In an effort to find some correlations, I found the following possible explanations, but would like to know for sure what's going on (I understand that a double count of imap is possible?):

maybe customer owned files in /home/customer/imap counted twice:
6645 4KB blocks
27217920 bytes
26.0 MB

or perhaps logs & stats:

/home/customer/domains/domain/logs:
5313 4KB blocks
21762048 bytes
20.8 MB

/home/customer/domains/domain/stats:
564 4KB blocks
2310144 bytes
2.2 MB
 
Last edited:
It's also worth mentioning that quota appears to be reporting 1KB blocks, though this isn't technically correct as per the filesystem format options stipulated by tune2fs. Regardless of whether I count up 1KB block or 4KB block, I'm always a bit shy.

First, issued a recount/rebuild:
Code: 
/sbin/quotaoff -a; /sbin/quotacheck -augm; /sbin/quotaon -a
Then got the quota output for the customer:
Code: 
quota -vl customer
Disk quotas for user customer (uid not 0): 
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
      /       0       0       0               0       0       0        
      /home  125280  256000  256000            2118       0       0

Lastly, issued the respective find's for all customer owned files in /home/customer:
4KB = 31085 (or 4 * 31085 = 124340)
1KB = 124340

(note that the likelihood of a true 1KB "block" count matching a 4KB block count--unless running against the same underlying 4KB filesystem--is rather astronomical, so really a 1KB count is just a 4KB count / 4 to give a more human readable output...but I digress)

Obviously these are reporting values which are nearly 10MB off. I am nearly 100% certain that no files exist outside of the customer's home directory. How accurate is the linux quota reporting...? I ask because we've got some rather brutal directory structures in /home which make a /home wide find cli somewhat difficult.
 
Last edited:
Interesting option. However, the email location assumption is flawed. There are actually not 2 possible locations that email can be, but 3:
1) on / or /home (quota'd)
2) on /var (NOT quota'd)
3) on /home & /var (quota'd _AND_ not quota'd)

In other words, I need DirectAdmin to use quota for files that are on quota'd filesystems, and count them up on non-quota'd filesystems.

The problem really comes down to email being stored in 4 different places:
1) /home/customer/mail = admin IMAP
2) /home/customer/imap = user IMAP
3) /var/spool/mail = admin POP
4) /var/spool/virtual/customer_domain = user POP

Is there a reason that /var should not be mounted usrquota,grpquota, so that everything is quota'd?
 
Hello,

DA only counts 1 quotaed partition. When the reqpquota command is run (and setquota for that matter), only 1 partition is specified. If /var were quotaed as well, DA wouldn't set any limits because it would be on a seperate partition from the file in /home (or /, whatever you have setup). The user_create_post.sh script can be used to set the quota limit on another partition, but since DA only counts the 1 partition (which would be / or /home), it just manually counts the files in /var. (It does a check to see if /var is on the same partition as /home). If it is, it lets the quotas count it, if not, it manually counts each email file in /var.

The only advantage to adding quotas to /var would be to prevent an oversized inbox.

John
 
I'm not sure I understand.

1a) Are you stating that if /, /home & /var are all separate partitions, that DirectAdmin only uses quota to tally up files on /home?
1b) If so, is there any reason to have quota set on /?

2) More importantly, the root point of this entire topic is to determine why there is a descrepancy between counting the files manually, and DirectAdmin's customer disk usage tally. Please review and respond to the findings painstakingly detailed in my posts on 2005/07/03.

I greatly anticipate your response, and sincerely appreciate your diligence on this matter.

Regards,
-the donco
 
Hello,

1a) yes.
1b) no. DA only does it becaue some systems only have / and no other partitions. The installer just set / and /home with quotas to cover all bases. Normally, just /home is enough. (yes if you only have / and no /home)

2) One thing that comes to mind that I might have failed to mention is that DA also manually goes through /home/username/domains/* in search of any files owned by apache. This may be the cause of the discrepancy.

John
 
Thanks John,

I tried the following:
Code: 
find /home/customer -user apache
find /home/customer -uid `id -u apache`
No files were found with either command, so apache files aren't the problem.
 
You must log in or register to reply here.
Back
Top