Backup - enable multicore gzip

[zakazak]

I am currently doing a migration to a new DA Server by making the admin backup via ftp to the new server.
I notice that each user gets packed with gzip but only by using 1 CPU.

This seems like a bottleneck that makes backup/restore take ages.

Why not make it use multicore or e.g. 50% of available cores?

 

[zakazak]

Okay I noticed that one can install pigz (e.g. apt-get install pigz) and then set pigz=4 in directadmin.conf.
This should now use 4 cores whenever I create or restore backups?

 

[Stije]

You should use zstd, it is faster and uses less space.

And yes. You can do:

cd /usr/local/directadmin/custombuild
./build update
./build pigz

pigz=4 would tell pigz to use 4 processes.

 

[zakazak]

Ah thanks, I did that.
I wonder why it is not set like that by default?

 

[Zhenyapan]

@zakazak because nowadays default zst format.
earlier pigz was compilled, then replaced by package from OS. Also some vps 1-4 cores may stuck if pigz will use few cores.

 

[zakazak]

@zakazak because nowadays default zst format.
earlier pigz was compilled, then replaced by package from OS. Also some vps 1-4 cores may stuck if pigz will use few cores.

Srroy, I meant why isn't zstd enabled by default?

 

[Richard G]

why isn't zstd enabled by default?

That is what @Zhenyapan already wrote. Default is zst format so zstd is enabled by default nowadays.

Only pigz is not enabled by default, because not everybody has multiple cores.

 

[zakazak]

That is what @Zhenyapan already wrote. Default is zst format so zstd is enabled by default nowadays.

Only pigz is not enabled by default, because not everybody has multiple cores.

Oh okay, I misunderstood it then.

So zstd is enabled but multiple cores isn't.
How to enable multicore zstd then?

 

[Richard G]

So zstd is enabled but multiple cores isn't.

Correct.

There is no multicore zstd. The multicore is with pigz. When pigz is set, which is already shown by @Stije in post #3, then that amount of cores are used by DA during backup.

 

[johannes]

zstd must be compiled with multithread support, then it would automatically adjust the amount of threads, so far i have understood. Correct me if wrong. The big question is now - was it compiled with multithread support in DA?

 

[Richard G]

was it compiled with multithread support in DA?

It isn't compiled. It's installed via yum.