critical problem Back up one million folders with more than 5 files

A

AhrimanSefid

Verified User
Joined
Nov 2, 2016
Messages
19
Hi
I've faced a significant problem and I appreciate your help and instructions. I have more than one million folders with more than 5 files in each. I need to back up but it is not possible. Could you please guide me through the process?
Regards
 
Why isn't it possible? We backup servers with hundreds of millions files, but it takes a while and many small files use a lot of ram ( for caching directory indexes etc ) and most often use 1 or more 4K blocks on disk. (So if you have millions of 2k files, you're wasting 50% of your diskspace).
 
AhrimanSefid said:
Hi
I've faced a significant problem and I appreciate your help and instructions. I have more than one million folders with more than 5 files in each. I need to back up but it is not possible. Could you please guide me through the process?
Regards
Click to expand...

Hello, what error gives you? If it is by the indexes you can in directadmin.conf deactivate them
 
I need to increase the speed of back up process as much as possible. i only have one website which is located on a server with 32G RAM, CPU: e3-1270 and 2-4 TB HDD.
 
Perhaps you reached some kind of limit in your packer/backup program. Figure out the version number and investigate if there are any limits. If a newer version is available, try that one.
 
Hello,

Usually directadmin gives some details if a backup fails. If you don't them in Directadmin messages system, then you might consider checking:

/var/log/directadmin/error.log
/var/log/directadmin/errortaskq.log

if they do not provide details, then you might need to start backup manually with debug mode:

1. open ssh session with your server and type:

Code: 
/usr/local/directadmin/dataskq d8000 2>&1 | tee out.txt

but don't hit enter.

2. open directadmin and select to backup your user or choose to execute the backup now.

3. click enter in ssh console to start backup process with debugging

when it stops or breaks you might have an error message to analyze in console and/or in out.txt.
 
You must log in or register to reply here.
Back
Top