Compress mail manually (dovecot_compress.sh)

X

xerox

Verified User
Joined
Jul 16, 2019
Messages
145
Hello,

CustomBuild 2 have option to compress new emails, but i would like to know how to manually compress mail which is already existing in inbox and sent items.

I did not see any explanation how to do it manually.

Screenshot:
cmail.jpg

Ideas ?
Thanks a lot.
 
Hello Xerox,

You can run following script to compress existing mails/boxes:

/usr/local/directadmin/scripts/dovecot_compress.sh
 
Hey! Thanks for the info, i totally missed that file in that dir.
 
Hi all,

Am I correct the script only compresses files in the ./cur directory and not all other directories?
I use many subfolders for my mail and they ar stored like ./INBOX.folder1 /.INBOX.folder2 just next to ./cur.

I checked out the script and I expected all directories should be processed (and compressed), but ./cur is the only folder which is compressed after running the script.

I run the script using:
Bash: 
/usr/local/directadmin/scripts/dovecot_compress.sh /home/user/imap/domain.com/email/Maildir
The result is
Bash: 
Checking for directories in /home/user/imap/domain.com/email/Maildir...
Size before compression: 16M
Compressed /home/user/imap/domain.com/email/Maildir/./cur...
Size after compression: 6.1M

But when I do
Bash: 
du -h Maildir/
The result is something like the following:
Bash: 
4.0K    Maildir/.INBOX.IsGeenSpam/new
4.0K    Maildir/.INBOX.IsGeenSpam/tmp
4.0K    Maildir/.INBOX.IsGeenSpam/cur
20K     Maildir/.INBOX.IsGeenSpam
12K     Maildir/cur/tmp
6.1M    Maildir/cur
4.0K    Maildir/.Drafts/new
4.0K    Maildir/.Drafts/tmp
4.0K    Maildir/.Drafts/cur
20K     Maildir/.Drafts
4.0K    Maildir/.INBOX.Thermen/new
4.0K    Maildir/.INBOX.Thermen/tmp
672K    Maildir/.INBOX.Thermen/cur
692K    Maildir/.INBOX.Thermen
4.0K    Maildir/.INBOX.Sent/new
4.0K    Maildir/.INBOX.Sent/tmp
556K    Maildir/.INBOX.Sent/cur
576K    Maildir/.INBOX.Sent
4.0K    Maildir/.Archives.2010/new
4.0K    Maildir/.Archives.2010/tmp
12M     Maildir/.Archives.2010/cur
12M     Maildir/.Archives.2010
4.0K    Maildir/.Archives.2013/new
4.0K    Maildir/.Archives.2013/tmp
38M     Maildir/.Archives.2013/cur
38M     Maildir/.Archives.2013
4.0K    Maildir/.INBOX.Marktplaats/new
4.0K    Maildir/.INBOX.Marktplaats/tmp
13M     Maildir/.INBOX.Marktplaats/cur
13M     Maildir/.INBOX.Marktplaats
4.0K    Maildir/.INBOX.Facebook/new
4.0K    Maildir/.INBOX.Facebook/tmp
322M    Maildir/.INBOX.Facebook/cur
324M    Maildir/.INBOX.Facebook
4.0K    Maildir/.INBOX.Trash/new
4.0K    Maildir/.INBOX.Trash/tmp
4.0K    Maildir/.INBOX.Trash/cur
24K     Maildir/.INBOX.Trash

etc.
So there are many more folders next to ./cur, but they're not being compressed. Am I expecting the wrong result or should I compress every folder separately?

Regards,
Danny
 
Dannik said:
Am I correct the script only compresses files in the ./cur directory and not all other directories?
Click to expand...
That's right...
Dannik said:
So there are many more folders next to ./cur, but they're not being compressed. Am I expecting the wrong result or should I compress every folder separately?
Click to expand...
You can't without modify the script... it checks if you provide a path ending with /Maildir, and all folders inside your maildir folder doesn't contain that but just cur folder.

Also, the script doesn't delete the tmp folder created inside cur folder.

Regards,
Dan
 
@Dannik , @lordlex

I also started to use this script a few weeks ago and had some info from here
But indeed experiencing the same like you.

I'm busy to find out if I can make it better.
The first loop I want to change already to something like:
Code: 
find . -regextype egrep -regex '.*/(cur|new)' -print0 | while read -d $'\0' directory; do {
Like this I want to find all the cur and new folders.

If you change the script already on that line, I'm experiencing for the moment issues when the folder name has a space in it.
And something else I think I saw is that all emails are set to read state.
Once I make some progress I will share it.
 
wavoe said:
@Dannik , @lordlex

I also started to use this script a few weeks ago and had some info from here
But indeed experiencing the same like you.

I'm busy to find out if I can make it better.
The first loop I want to change already to something like:
Code: 
find . -regextype egrep -regex '.*/(cur|new)' -print0 | while read -d $'\0' directory; do {
Like this I want to find all the cur and new folders.

If you change the script already on that line, I'm experiencing for the moment issues when the folder name has a space in it.
And something else I think I saw is that all emails are set to read state.
Once I make some progress I will share it.
Click to expand...
Why not to just use ./dovecot_compress.sh all ? :)
 
smtalk said:
Why not to just use ./dovecot_compress.sh all ? :)
Click to expand...
Uhm I'm currently not looking at compressing all configured email addresses but I'm looking at having the email items *also in the (sub) folders*?
@smtalk if I'm correct this shell is only compressing the emails in the real inbox folder, and as an example not the Trash folder?
 
wavoe said:
Uhm I'm currently not looking at compressing all configured email addresses but I'm looking at having the email items *also in the (sub) folders*?
@smtalk if I'm correct this shell is only compressing the emails in the real inbox folder, and as an example not the Trash folder?
Click to expand...
It's compressing all the mails in all the folders.
 
Thanks @smtalk , I didn't know all was an option ?. The shell script doesn't mention it in the usage...
Bash: 
#VERSION=0.0.2
# This script is written by Martynas Bendorius and DirectAdmin
# It is used to gzip all emails in Maildir directory
# Official DirectAdmin webpage: http://www.directadmin.com
# Usage:
# ./dovecot_compress.sh </home/user/imap/domains/domain.com/email/Maildir>

So do I cd into /home/user/imap/domain.com/email/Maildir
and then run:
/usr/local/directadmin/scripts/dovecot_compress.sh all
?

or just run
./dovecot_compress.sh /home/user/imap/domains/domain.com/email/Maildir all


I'll be awaiting the results of @wavoe :)
 
Dannik said:
/usr/local/directadmin/scripts/dovecot_compress.sh all
Click to expand...

Code: 
# ./dovecot_compress.sh 
Usage:
./dovecot_compress.sh /home/user/imap/domains/domain.com/email/Maildir
or
./dovecot_compress.sh all
you gave #0: ./dovecot_compress.sh

It's mentioned :)
 
I believe you, but I also think there must be a newer script circulating then what's available on my server... :rolleyes:

Bash: 
if [ $# -lt 1 ]; then
        echo "Usage:";
        echo "$0 /home/user/imap/domains/domain.com/email/Maildir";
        echo "you gave #$#: $0 $1";
        exit 0;
fi

Bash: 
[root@beta Maildir]# /usr/local/directadmin/scripts/dovecot_compress.sh
Usage:
/usr/local/directadmin/scripts/dovecot_compress.sh /home/user/imap/domains/domain.com/email/Maildir
you gave #0: /usr/local/directadmin/scripts/dovecot_compress.sh
 
Ah well spotted, I see I have suddenly version 0.0.3 now :LOL:

@Dannik I recently did the update of DA to 1.60
When I started testing I also had this 0.0.2 version.
 
Ok. That must be the thing then.
I will wait a week or so before updating. Like now, I have seen before that after releasing an update, 1 or more patches are released quite quickly to resolve some bugs. It's normal, I know (I work for a software developing company and bugs are quite common), but I want to limit the risks my customers suffer from those bugs ?
Besides that, I won't have the time the next few days to test the compression anyway...
 
smtalk said:
It's compressing all the mails in all the folders.
Click to expand...
I'm not yet that much further yet but to give an update:
I created again a test account with some emails, mainly the biggest folders are inbox itself and the sent folder.
Using the dovecot_compress.sh (v0.0.3) is not compressing "all" the folders as I was expecting.

So on the left the Maildir folder before using the dovecot_compress.sh, on the right when it finished.
1580940038206.png

And just a detail: Size before 1.2GB (which is 721 MB + the tmp folder with compressed email of 471 MB) :cool:

Just sharing my experience now, if I have a free moment I would like to deep digger into the dovecot_compress.sh
 
If I'm not mistaking you forgot the all option, like smtalk mentioned...? However I'm still not sure if this works the selected mailbox or al mailboxes on the server. Haven't had any time to figure that out myself ?
 
Dannik said:
If I'm not mistaking you forgot the all option, like smtalk mentioned...? However I'm still not sure if this works the selected mailbox or al mailboxes on the server. Haven't had any time to figure that out myself ?
Click to expand...
@wavoe I think so too :) It wasn't "all" option, was it? If not, you just compressed the mail folder you specified, and it worked as intended.
 
smtalk said:
@wavoe I think so too :) It wasn't "all" option, was it? If not, you just compressed the mail folder you specified, and it worked as intended.
Click to expand...
A part of the dovecot_compress.sh:
Bash: 
if [ "${1}" = "all" ]; then
        cat /etc/virtual/*/passwd | cut -d: -f6 | sort | uniq | while read line; do {
                doCompressMaildir "${line}/Maildir"
        }
        done
else
        doCompressMaildir "${1}"
fi

For me this means that using "all" option will compress all email accounts that exists on the server.
And it's just like I'm doing manually, placing the location of the Maildir folder.
Meaning that I come back on the issue what I mentioned earlier.

And just an example of the output of cat /etc/virtual/*/passwd | cut -d: -f6 | sort | uniq
Code: 
...
/home/testemail1/imap/testemail1.com/info
/home/testemail2/imap/testemail2.com/info
/home/testemail3/imap/testemail3.com/info
/home/testemail4/imap/testemail4.com/info
/home/testemail5/imap/testemail5.com/info
...
This is not what I want (yet), I just want to be sure that every email folder is getting compressed.
Hope someone can confirm what I describe here as I have mixed feelings as it's not the first time that I try to explain this.
 
wavoe said:
A part of the dovecot_compress.sh:
Bash: 
if [ "${1}" = "all" ]; then
        cat /etc/virtual/*/passwd | cut -d: -f6 | sort | uniq | while read line; do {
                doCompressMaildir "${line}/Maildir"
        }
        done
else
        doCompressMaildir "${1}"
fi

For me this means that using "all" option will compress all email accounts that exists on the server.
And it's just like I'm doing manually, placing the location of the Maildir folder.
Meaning that I come back on the issue what I mentioned earlier.

And just an example of the output of cat /etc/virtual/*/passwd | cut -d: -f6 | sort | uniq
Code: 
...
/home/testemail1/imap/testemail1.com/info
/home/testemail2/imap/testemail2.com/info
/home/testemail3/imap/testemail3.com/info
/home/testemail4/imap/testemail4.com/info
/home/testemail5/imap/testemail5.com/info
...
This is not what I want (yet), I just want to be sure that every email folder is getting compressed.
Hope someone can confirm what I describe here as I have mixed feelings as it's not the first time that I try to explain this.
Click to expand...

Sorry, you're right :) Double-checked the code and found the bug. Please check if the following 0.0.4 version does the trick (it should):
Code: 
wget -O /usr/local/directadmin/scripts/dovecot_compress.sh https://custombuild.eu/dovecot_compress.sh

We could include it into the next release of DA then.
 
You must log in or register to reply here.
Back
Top