We have noticed lately that making backups is very slow Backup - enable multicore pigz does it still work if?

[Niels90]

We've recently noticed that backups are very slow. Backup - enable multicore Pigz. Does it still work when we run the commands /usr/local/directadmin/custombuild and ./build? Pigz doesn't recognize DirectAdmin. Pigz is installed on the system. We're using zstd in the DA configuration.

 

[zEitEr]

Hello,

Still in play I believe:

- https://docs.directadmin.com/changelog/version-1.59.0.html#pigz-can-be-managed-by-system

I have:

[root@poralix ~]# da v
DirectAdmin 1.690 ddfdf9c5355920dcb37b94ac76d8f2527da2d7a4
[root@poralix ~]# da c | grep pigz
pigz=1
pigz_bin=/usr/local/bin/pigz
unpigz_bin=/usr/local/bin/unpigz
[root@poralix ~]#

 

[Niels90]

Hello,

Still in play I believe:

- https://docs.directadmin.com/changelog/version-1.59.0.html#pigz-can-be-managed-by-system

I have:

[root@poralix ~]# da v
DirectAdmin 1.690 ddfdf9c5355920dcb37b94ac76d8f2527da2d7a4
[root@poralix ~]# da c | grep pigz
pigz=1
pigz_bin=/usr/local/bin/pigz
unpigz_bin=/usr/local/bin/unpigz
[root@poralix ~]#

we have this web01:~# da v
DirectAdmin 1.690 ddfdf9c5355920dcb37b94ac76d8f2527da2d7a4
root@web01:~# da c | grep pigz
pigz=8
pigz_bin=/usr/bin/pigz
unpigz_bin=/usr/bin/unpigz
root@web01:~#

does that work with zstd or not or only with tar.gz or 0 .tar file will be created as a backup
1 .tar.gz file will be created as a backup
2 .zstd file will be created as a backup whichever of the 3 ?

 

[zEitEr]

pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data.

zstd - Facebook developed realtime compression algorithm. Multi-threading supports up to 256 threads

related:

- https://docs.directadmin.com/changelog/version-1.46.0.html#pigz-for-parallel-process-backup
- https://community.centminmod.com/th...d-vs-brotli-vs-pigz-vs-bzip2-vs-xz-etc.17259/

 

[johannes]

3 questions came up:

1) according to https://docs.directadmin.com/changelog/version-1.59.0.html#pigz-can-be-managed-by-system : "./directadmin set pigz 4" .. "where 4 is the number of threads you want".
So I would assume more threads is equal to faster compression. But on the table on https://community.centminmod.com/th...d-vs-brotli-vs-pigz-vs-bzip2-vs-xz-etc.17259/ it seems to go the opposite direction. Higher level = slower compression. Or is this so called pigz level there in the table another thing? If its another thing, then how to set it with directadmin command?

2) Is zstd-level connected with how many threads are used? Or is the level another thing, and its using automatically available cpu threads?

3) So far i understand, zstd is the better way, am I right?

 

[zEitEr]

3 questions came up:

Johannes, thank you for your input. I did not dive that deep in the question yet. My intention was to show PIGZ and ZSTD are different things and they go independently. Probably somebody else could join the discussion and answer in details.