Backups are crashing when quota exceeded

A

ActiveDomain.NL

Verified User
Joined
Sep 24, 2003
Messages
23
Location
Rotterdam, the Netherlands
Hello,

There is a problem that hasn't been fixed in the past few months. When users are using the backup function and during the backup, their quota is exceeded, the tar program starts using 100% of CPU. Most of the users restart the backup, because the backup function doesn't respond anymore, causing a really high load on the server, because 2 or 3 tar processes are using 100% CPU!

For example:
- User website is 75 MB
- User quota is 100 MB
This will reproduce this problem, because after 25 MB the user exceeded his quota and so the tar process will use 100% CPU.

I have this problem on FreeBSD 4.x and 5.x. I don't know if Linux has the same problem?

So the question is: how can I fix this? Is it maybe possible to add some check, so that a user can start only 1 backup process? Should be easy: save the PID of the first backup process and if the user wants to run another backup, check whether that PID is running or not.

Any help is welcome :)
 
I have to set quota's. I'm in the hosting business for 5 years now, but my experience is that if I don't use quota's, people are simply going to use a few GB's of storage sometimes. And in a few cases this can cause problems, because there isn't enough space left on the partition/disk.

Does anyone have an idea how to really solve this issue?
 
What do you want them to fix?

It sounds like you want to set quotas yet have them ignored.

How would you recommend they fix that?

Jeff
 
ActiveDomain.NL said:
I have to set quota's. I'm in the hosting business for 5 years now,
Click to expand...
I'm not sure if it matters but I've been in the hosting business since 1994. :confused:
but my experience is that if I don't use quota's, people are simply going to use a few GB's of storage sometimes.
Click to expand...
If your drives aren't large enough to handle the occasional 24 hours worth of overage you're absolutely correct, but the server should be sending you messages overnight if anyone is over quota.
Does anyone have an idea how to really solve this issue?
Click to expand...
Okay, let's visit this:

You want to set quotas.

But you want the backup program to ignore them.

There are two ways to handle this problem.

One is to have all backups created by the root. Then users couldn't manage their own backups.

Another is to have the backups created on another partition; one that doesn't have quotas enabled. This could be done but requires an extra level of complexity.

If you require this couldn't you set it up yourself with partition management, directory management, and links?

Jeff
 
The harddisks are large enough to handle a overage of 10-20 GB, but when I disable quota's, some users seem to like uploading complete DVD movies to the servers. Of course it is nice to have warnings, but it costs a lot of time to disable users, answer support mails, etcetera.

I don't want the backup program to ignore quota's. I want the backup program to do one of the following things:
- Give an error before creating the backup, so I don't have to handle all the e-mails from clients complaining that somehow their backups are not created and I don't have to kill the 100% CPU processes
- Give users the possibility to simply add or remove things from the backup they want to run, with details about how much space it uses, so they can still create a backup.

I already have nightly backups to a backup server, but some users also want to make a backup of their site, for example when they are going to upgrade their CMS or forum and they want to be able to downgrade when something goes wrong.
 
The backup program has no idea in advance how much space a backup will take.

I suppose it could base the warning on how much space is being backed up, but I'm not sure that would fit your needs.

Have you posted this as a feature request?

Jeff
 
I think DA puts backups in a tmp dir owned by diradmin or admin so it wont use the user's hd quota, at least thats what I thought.
 
Quota usage isn't determined by directory but rather by file ownership.

If the partition has quotas enabled, and if the user who owns the file has a quota, the quota is enforced.

On both Linux and Unix.

Jeff
 
jlasman said:
Quota usage isn't determined by directory but rather by file ownership.

If the partition has quotas enabled, and if the user who owns the file has a quota, the quota is enforced.

On both Linux and Unix.

Jeff
Click to expand...

so here is what I think is fair:
the backup get generated on a DA temp dir, owned by diradmin (or whichever use runs DA)
after the backup is done, DA see if backup+(user used quota) <= (user quota)

if so, just change ownership to user and send the file to his homedir
else, error

another possible thing would be to just allow backups if user has at least x% (maybe 40) of his space left
 
Having thought about this it is very unfair to the user to expect them to only use half of their space so they can run the backup tool, I think the backup file should be owned by diradmin and the amount of backups a user can have sitting on the server at once settable by the admin so if its eg. 2 then the user can have 2 seperate backups at any time which wouldnt affect their hd quota.

An alternative is to use soft quotas on top of hard quotas allowing them to go over the limit for a grace period which should allow backups to work.
 
Unfair or not, this is not the place to request a change to DA.

There's a specific forum for that.

:)

Jeff
 
jlasman said:
Unfair or not, this is not the place to request a change to DA.

There's a specific forum for that.

:)

Jeff
Click to expand...

it's not exactly a request at this time
I consider we're just discussing what would be the best solution
 
You must log in or register to reply here.
Back
Top