Compress mail manually (dovecot_compress.sh)

smtalk said:
Sorry, you're right :) Double-checked the code and found the bug. Please check if the following 0.0.4 version does the trick (it should):
Code: 
wget -O /usr/local/directadmin/scripts/dovecot_compress.sh https://custombuild.eu/dovecot_compress.sh

We could include it into the next release of DA then.
Click to expand...
Ok thanks for your response and quick customization of the script.
I did the test again today with the same start data (referring to my previous screenshot), the result is now as following:
* the other mailbox folders are taken and the compression is happening however it seems to be that the compressed emails in the tmp folder are not moved (tmp folder contains cur and new folder).
* The size before is now correct in this version.
1581028744560.png
 
wavoe said:
Ok thanks for your response and quick customization of the script.
I did the test again today with the same start data (referring to my previous screenshot), the result is now as following:
* the other mailbox folders are taken and the compression is happening however it seems to be that the compressed emails in the tmp folder are not moved (tmp folder contains cur and new folder).
* The size before is now correct in this version.
View attachment 2931
Click to expand...

Aren’t cur/ and new/ empty in tmp? :)
 
Tested 0.0.4 and found that converted files left in tmp. So, the space used is increased since it combine both uncompressed and compressed files.
 
nmb said:
Tested 0.0.4 and found that converted files left in tmp. So, the space used is increased since it combine both uncompressed and compressed files.
Click to expand...
Bug found/fixed. tmp can be safely removed. New script will clean it up :) To update, just use:
Code: 
wget -O /usr/local/directadmin/scripts/dovecot_compress.sh https://custombuild.eu/dovecot_compress.sh

It'll be included in DA 1.60.2.
 
smtalk said:
Bug found/fixed. tmp can be safely removed. New script will clean it up :) To update, just use:
Code: 
wget -O /usr/local/directadmin/scripts/dovecot_compress.sh https://custombuild.eu/dovecot_compress.sh

It'll be included in DA 1.60.2.
Click to expand...
After testing 1 mailbox it seems to be that all email folders are now compressed as I was expecting.
Thanks for having this solved!

Maybe one last remark concerning the docs / comments, the example path contains the folder "domains".
I mean now you say "/home/user/imap/domains/domain.com/email/Maildir" and maybe it's better to change it to "/home/user/imap/domain.com/email/Maildir" to avoid some confusing.
 
Tested a few email accounts and it looks like the compression script will create another extra blank email in the end of the mailbox. (Check 3 accounts I owned and each account has a new blank email in as the latest mail.)
 
nmb said:
Tested a few email accounts and it looks like the compression script will create another extra blank email in the end of the mailbox. (Check 3 accounts I owned and each account has a new blank email in as the latest mail.)
Click to expand...
If I check the test mailbox it seems to be that I don't have this issue in fact.
Will try to launch some email counting before and after later to be sure.

If you search for the email file, is it a file of 0 bytes or?
 
Email file is not 0 byte. Below is the info I got :

-rw------- 1 xxxxx xxxxx 45 Feb 7 17:52 1581075140.M966140P28241.xxxxx.xxxxx.com,S=25,W=25:2,Sa
 
I think I found the reason why I got this new blank email. When compress is done, if nothing show, I will not get blank email but if I get warning similar to below, blank email will appear :

Warning: Our dotlock file /home/xxx/imap/xxx.xxx/xxx//./Maildir/cur/dovecot-uidlist.lock was deleted (locked 2 secs ago, touched 2 secs ago)
Click to expand...
 
nmb said:
I think I found the reason why I got this new blank email. When compress is done, if nothing show, I will not get blank email but if I get warning similar to below, blank email will appear :
Click to expand...

Hm.. I think it might not like the lock in cur directory and renames it then. May you try this again:
Code: 
wget -O /usr/local/directadmin/scripts/dovecot_compress.sh https://custombuild.eu/dovecot_compress.sh

That empty file can be safely removed, as it's just a copy of dovecot-uidlist.lock.
 
I don't see that warning anymore while compressed quite a lot of email accounts until I just done with all of them.

Thank you for the script. That save me a lot of disk space :)
 
I ran the dovecot_compress.sh today on a real account and the email size went from 3.5 GB to +- 2 GB.
This is also what I see in DA itself however in roundcube the 3.5 GB is shown.
Running doveadm quota recalc -u [email protected] does not change the size and doveadm quota get -u [email protected] just shows the uncompressed size.

In fact I was expecting that the maildirsize would also shrink.
Is this just working as designed or someone else experienced this and has some solution?
I'm not really a fan to just increase the mailbox size and set it higher than the account disk limit for example..
 
Yeah great to see that you already went in discussion with them. I'm not really "surprised" to say it like this.
Nice to see that you also proposed a patch, maybe something to consider to have a custom version then..
 
It would be nice if there is an option to compress emails older than X (for example 90) days and make a daily cronjob to compress emails getting older than X days.
This way you still safe disk space but performance does not hurt that much since old emails are not opened that much anymore.

How do you think about this idea?
 
The best place for feedback is in the feedback site.
myH2Oservers said:
It would be nice if there is an option to compress emails older than X (for example 90) days and make a daily cronjob to compress emails getting older than X days.
This way you still safe disk space but performance does not hurt that much since old emails are not opened that much anymore.

How do you think about this idea?
Click to expand...
 
You must log in or register to reply here.
Back
Top