Logic to prevent system backup from filling up hard drive

keefe007

Verified User
Joined
Dec 25, 2004
Messages
282
I think the system backup is missing a critical feature, especially due to the fact that DA overwrites config files with 0 byte files when the disk is full. Some simple logic should be included to skip the backup and notify the admin if the backup would have filled the hard drive.

What do you think DA people?
 
There are already a few measure in place:

1) Admin Level -> Administrator Settings -> The bottom setting: "Warn Admin's at X% partition usage. Check every Minute/Day/Never

2) The System Backup has a built in disk space checking for the partition you set. You can specify how much free space you want to have in order for it to run. Less than that number, it wont run. (the partition has to be correctly selected)

3) We do have checks in place for the config files, but it's apparent that more checks are needed. Every single byte that DA sends to a file in a config file is counted. The write process will return how many bytes are correctly written. This is the simple logic that we want. The problem is, this is complelely inaccurate as it seems to always return the correct number, even if nothing is being written. We're guessing this has to do with some sort of file system caching before writing for optimized disk access. We'll probably add a simpler check of looking at the filesize after it's done (flushed/closed). At the moment, DA does write to a 2nd file first, then move it in place of the original. This can still be used, but a "final disk size check" is likely the more reliable check (before the rename).

John
 
We're guessing this has to do with some sort of file system caching before writing for optimized disk access. We'll probably add a simpler check of looking at the filesize after it's done (flushed/closed).
Of course. I think every Posix-style system optimizes disk access; I know that FreeBSD (and the other BSDs) and Linux both do. It's in the kernel, and would require a change to the kernel to make it work otherwise ... and of course it would slow down the system significantly, because heads are almost never where they need to be to optimize a disk write.
At the moment, DA does write to a 2nd file first, then move it in place of the original. This can still be used, but a "final disk size check" is likely the more reliable check (before the rename).
Are you using the equivalent of an mv command, John?

If so consider that mv command works differently depending on whether or not the file is being moved to the same partition.

If to the same partition, then mv simply changes the directory listing, then releases the space used in the old listing.

But if to a different partition, then mv works as a copy and then a delete. While I'm not sure exactly in what order a copy does things, I do know that a copy is not failsafe in the case of a full drive, so I'm guessing it deletes first, then writes, not necessarily to the same space, then changes the directory listing to point to where it just wrote.

So while creating a temporary file first sounds great, and while it proves there was space to do it when you do it, if the permanent file isn't in the same partition as the temporary file, then it doesn't prove there'll be space left for the permanent file (if for example, something fills up the drive between delete and the write.

Or of course it could write first, and then delete, but not know how to abort if it can't write.

Does anyone care to spend time chasing this down? I just tried and couldn't easily find the documentation.

Jeff
 
The file is created right beside the original file. It will always be on the same partition.

Doing a quick stat on the temp file after writing to get it's size isn't that hard to do. I'll add it for the next release.

John
 
Another suggestion:


Keep trailing X days.

Wanna have 4 days back of history?
or maybe a single weekly or Two weekly copies?
Perhaps even a single Monthly? (I can't think of anyone that would need beyond a single monthly back).

But a trailing week or trailing weekly (keep one copy that is always seven days back from Today();).

-Alon.
 
It did it ???

Hi folks...

Gone away on holiday for a week, i had forgotten to warn the noob replacing me that on one server running DA, we were a bit low on space and that backups had to be removed during the week...
In result partition ran low on disk space, and i even had sunday a O Mb left on the home partition....

Why is DA not already able to cope backup rotation ???? Why is there no automatic deleting of old backups base on day - X setting ??? This is getting me nuts that this feature has not been yet asked in 3 years i use DA. And not even thought of by directadmin teams... Do you never run out of space
when using this feature ... (I rotate backups each week, by deleting each mondays w-1 backups)

Do something, or give us please a solution in cron commands so that linux noobs and learners can manage these backups without fearing a system disk space at 0 problem....

And i know google is my friend, i'm pretty sure i'll find as usual a faster solution such as a cron script or other... But this has gotten me a bit anger after DA...
A simple script could have been thought of by da team...

This is my 2 pence thought of the day

Tdldp
 

What can i say else than thank U ???? This is perfect, and i must admit that if i had done a correct search (for example going on page 2 on searching "delete backups" i'd have found the threads concerning my needs...weird that the threads are on page 2 and that nothing relevant appear on page one to the problem meeted)

Next time i'll be more careful, and search a bit more

Nevertheless, there should be something planned by the DA team on this in the backup features with a simple N° of days rotating function .... i'm going to read john's response to my mail, but i'm pretty sure there is nothing on the way on this problem.....
 
Back
Top