HOWTO: Use Dovecot's deliver & report quota through imap

aphexer

Verified User
Joined
Oct 18, 2009
Messages
12
Quick summary: Make Dovecot send a "imap notify" to your MUA when new mail arrives and make dovecot report quota usage through imap.

These are the problems I'm trying to solve:

A: When a mail arrives for a virtual user or a system user, exim uses the appendfile driver to put the mail in the user's Maildir. This driver just puts the mail there, without notifying dovecot there is new mail.

When a user has his MUA open, he will not be notified of new mail because dovecot doesn't know there is new mail. The user will see new mail when he manually clicks the "get mail" button or equivalent, triggering his MUA to explicitly ask the status of that mailbox. That triggers dovecot to scan the mailbox checking for new mail. (Note: it's possible to configure your MUA to do this automatically every 5 minutes or so).

I don't like this delay, we can do better. Instead of using exim's appendfile, I want to use dovecot's deliver, which also puts the email in the right place, but also notifies the dovecot imap process there is new mail. The MUA will instantly be notified of the new mail.

B: It's quite difficult to find out how much space you're using of your mailbox. If the admin has setup the DA squirrelmail plugin, then he can find out by logging in to squirrelmail webmail. But if you don't use or like squirrelmail you're quite of luck there. Only other option he has is to login in the DA control panel itself, but usually the end-mail-user does not have the login for the account itself, only for the mail account.

The IMAP protocol supports reporting quota usage to your MUA. That way I can see my quota usage from within my MUA and nearly all decent webmail interfaces will also report the quota usage, because they get it the same way: through IMAP.

Here is how to do it:

Step 1: configure dovecot

I only list the configuration pieces you have to modify, not the whole config.

Edit /etc/dovecot.conf:

Code:
protocol imap {
  mail_plugins = quota imap_quota
}

protocol pop3 {
  mail_plugins = quota
}

auth default {
  socket listen {
    master {
      path = /var/run/dovecot/auth-master
      mode = 0600
      user = mail # User running deliver
    }
  }
}

protocol lda {
  postmaster_address = [email protected]
  hostname = yourhost.example.com
  auth_socket_path = /var/run/dovecot/auth-master
  mail_plugins = quota
}

plugin {
  quota = maildir
  quota_warning = storage=95%% /usr/local/bin/quota-warning.sh 95
  quota_warning2 = storage=80%% /usr/local/bin/quota-warning.sh 80
}

Next create the /usr/local/bin/quota-warning.sh script which is referenced from /etc/dovecot.conf:

Code:
#!/bin/bash

PERCENT=$1
FROM_SMTP="[email protected]"
FROM="Support <[email protected]>"
qwf="/tmp/quota.warning.$$"

echo "From: $FROM
To: $USER
Subject: Your mailbox usage is $PERCENT% full
Content-Type: text/plain; charset="UTF-8"

Dear customer,

Your mailbox is $PERCENT% filled. Please remove some messages to
make sure new mail keeps arriving. When your mailbox is completely
filled (100%) no new mail will arrive. You can contact us at
[email protected] with your questions.

YourCompany" > $qwf

cat $qwf | /usr/sbin/sendmail -f $FROM_SMTP "$USER"

Don't forget to chmod +x /usr/local/bin/quota-warning.sh.

Step 2: Configure exim

Add the new dovecot transport in /etc/exim.conf:

Code:
dovecot_virtual_delivery:
  driver = pipe
  command = /usr/libexec/dovecot/deliver -d $local_part@$domain  -f $sender_address -a $original_local_part@$original_domain
  message_prefix =
  message_suffix =
  delivery_date_add
  envelope_to_add
  return_path_add
  log_output
  user = mail
  temp_errors = 64 : 69 : 70: 71 : 72 : 73 : 74 : 75 : 78

Now you want exim to actually use this transport. Find the virtual_user director and modify it to look like this:

Code:
virtual_user:
  driver = accept
  condition = ${perl{save_virtual_user}}
  domains = lsearch;/etc/virtual/domainowners
  group = mail
  retry_use_local_part
  transport = dovecot_virtual_delivery

You can do the same for the localuser director, but I'm not using that one here.

Step 3: Configure Directadmin

Now dovecot needs to know what the quota is that you set during email account creation. Create /usr/local/directadmin/scripts/custom/create_email_post.sh.

Code:
#!/bin/bash

umask 0077
#env > /tmp/env.log

function error
{
  MSG="To: [email protected]\nSubject: Error in $0\n\nError occured running $0. Error was: $1\n"
  echo -en $MSG | /usr/sbin/sendmail [email protected]
  exit 1
}

[[ "$quota" == "" ]] && error "quota"
[[ "$username" == "" ]] && error "username"
[[ "$domain" == "" ]] && error "domain"
[[ "$user" == "" ]] && error "user"

newquota=$((quota*1024*1024))

MAILDIR=/home/$username/imap/$domain/$user/Maildir
FILE=$MAILDIR/maildirsize
mkdir -p $MAILDIR
echo "${newquota}S,0C" > $FILE
chown -R $username:mail $MAILDIR || error "chown"
chmod 660 $FILE || error "chmod"

exit 0

In case you update the quota of a user, you need to rewrite the maildirsize file. Put this in /usr/local/directadmin/scripts/custom/email_change_pass_post.sh:

Code:
#!/bin/bash

env 0077
#env > /tmp/env.log

function error
{
  MSG="To: [email protected]\nSubject: Error in $0\n\nError occured running $0. Error was: $1\n"
  echo -en $MSG | /usr/sbin/sendmail [email protected]
  exit 1
}

[[ "$quota" == "" ]] && error "quota"
[[ "$username" == "" ]] && error "username"
[[ "$domain" == "" ]] && error "domain"
[[ "$user" == "" ]] && error "user"
[[ "$newuser" == "" ]] && error "newuser"

newquota=$((quota*1024*1024))
MAILDIR="/home/$username/imap/$domain/$user/Maildir"
FILE="$MAILDIR/maildirsize"
NEWMAILDIR="/home/$username/imap/$domain/$newuser/Maildir"
NEWFILE="$NEWMAILDIR/maildirsize"

[[ -d "${NEWMAILDIR}" ]] || error "maildir"

echo "${newquota}S,0C" > $NEWFILE
# Force quota recalculation
echo "-1024 -1" >> $NEWFILE
chown $username:mail $NEWFILE || error "chown"

exit 0

Don't forget to chmod +x both scripts and chown diradmin:diradmin.

What do you guys think? My customers are happier at least :) I'd love to see this become the default DA setup...

Note: Suggestions from comments are constantly being incorporated in the code snippets above.

23 Apr 2010: Added mail_plugins = quota in POP3 section (yeah some people still use POP3!)
 
Last edited:
What do you guys think? My customers are happier at least :) I'd love to see this become the default DA setup...
The first question, brought up by user scsi is simply "is there a problem?"

Has anyone else seen this problem. With what MUA do you see the problem? With desktup MUAs using IMAP, or with webmail IMAP clients?

Once we verify a problem, then hopefully a few people will start using this and let us know how it is or isn't working for them.

Once we see it works, and solves a real problem, I've got no problem coordinating changes with DirectAdmin staff and my exim.conf files to create a new standard.

Note that the new SpamBlocker Version 3-bet and above do NOT work without Dovecot, so this may be a good time to consider moving forward with this if it's really helpful.

Jeff
 
I never seen this problem.

First of all, I'm solving 2 problems. One is the imap notify and the other is the quota reporting.

I suppose you refer to the imap notify, regarding that: how often does your MUA check for new mail? If it polls your mail server every 5 seconds, sure you won't notice the difference.

This is how I'd demonstrate the problem: set your MUA check interval to some reasonable default if you changed it before (eg set it to 10 minutes).

Then send an email to that IMAP account and keep watching your MUA until it shows you that email. How long does it take? It'll be the check interval / 2 on average.

That's no problem for many people, it is for some. Even if it's no problem, why not optimize this delay away? You don't loose anything with it. You win time.

I prefer to see my mail when it arrives :)

For the second problem (the quota thing): try to find out how much disk space you used of your imap account without entering the DA control panel username/password. Hard isn't it? The only place you can see outside DA is through the DA squirrelmail plugin. My enhancement lets your MUA/webmail ask it through dovecot.
 
Last edited:
Who cares no email is important enough where you cant wait 10 mins lmao. Why do you have email that is so critical you must have it immediately?
 
Who cares no email is important enough where you cant wait 10 mins lmao. Why do you have email that is so critical you must have it immediately?

Sometimes it happens you have a conversation by email. You ask someone a question, they respond. You want to reply again and so on, kind of like instant messaging. That doesn't work when you have to wait 10 min for each mail to appear.

Why not let the trains wait in each station an extra 10 minutes? That way for sure trains will never be late, as they can catch up with their potential delay in each station. But who the hell wants to wait 10 minutes at each station? It makes going from point A to B take a lot longer. It's the same thing.
 
hahaha so silly ... all you have to do is lower the time it checks for mail. If you want an instant messenger then use that. If you need your mail instantly then just change it to check 1 minute. :p
 
hahaha so silly ... all you have to do is lower the time it checks for mail. If you want an instant messenger then use that. If you need your mail instantly then just change it to check 1 minute. :p

10000 imap users sending a check every minute is about 30 bytes * 10000 = 300 kbytes/sec constant rate, just to check if there is mail? That's 25GB a day.

If a user only receives about 1 mail per hour, and you use the imap notify command the same amount of users requires only 7 MB per day. Huge difference. 10000 users is too much for you? Just lower it then, it stays a factor 3600 between the two values.

Secondly the load on the mail server is a lot higher when you let every user poll the server that much. Using imap notify you remove the need to poll that often.

But let's reverse the question: why are you against my proposal? You think it will affect performance in a bad way? Things only get better. I claim they get a lot better, you claim it doesn't make much of a difference. At least it's better.

Also, what's your solution to the quota problem? You seem to focus on one problem only, while I'm solving two.

Btw: why do you think the imap notify extension was invented? There was a real problem requiring a real solution.
 
Updated email_create_post.sh

I updated the email_create_post.sh script to create the files with the correct permissions:

Code:
#!/bin/sh

umask 0077
#env > /tmp/env.log

function error
{
  MSG="To: [email][email protected][/email]\nSubject: Error in $0\n\nError occured running $0. Error was: $1\n"
  echo -en $MSG | /usr/sbin/sendmail [email][email protected][/email]
  exit 1
}

[[ "$quota" == "" ]] && error "quota"
[[ "$username" == "" ]] && error "username"
[[ "$domain" == "" ]] && error "domain"
[[ "$user" == "" ]] && error "user"

MAILDIR=/home/$username/imap/$domain/$user/Maildir
FILE=$MAILDIR/maildirsize
mkdir -p $MAILDIR
echo "${newquota}S,0C" > $FILE
chown -R $username:mail $MAILDIR || error "chown"


exit 0
 
Hello,
I've never had any problem with the current Exim config in regard to not having instant access to my emails.
The second they reach the server I get them in my Inbox and on my mobile devices.
For that you need to use the IMAP IDLE feature in your client (i.e. Thunderbird).

Regarding the quota setup. I was looking for a solution and will test this and report back :). We're using Horde and it supports maildir quota, so it should work.

Cheers,

Olivier
 
email_change_pass_post.sh is generating errors because I think it's not checking for the existence of the quota file before trying to change it.
 
Thanks for taking a look. I updated the email_change_pass_post.sh script in my first post to fix this.

Can you confirm it works now?
 
error

Hello,

I tried it and got this error in the exim paniclog:
Exim configuration error in line 672 of /etc/exim.conf:
router dovecot_virtual_delivery: cannot find router driver "pipe"

This is the line 672:
driver = pipe

(Freebsd 7.2 x64)
 
Do you have SpamAssassin installed? If you do, make sure you've got the right section(s) uncommented.

The reference SpamBlocker-powered exim.conf file supplied by DirectAdmin may be found here; the reference for SpamBlocker-powered exim.conf version 3-beta supplied by me may be found here.

Jeff
 
I have SpamAssasim installed and the the reference SpamaBlocker 2.1.1 exim.conf installed and uncommented and it is working fine.

Abraham
 
Old thread, but I just wanted to say that I've tested it and that it's possible to set IMAP quota that way.
The email_change_pass_post.sh seems wrong to me with all the operations about newuser, newfile, etc., but if you remove all that, it works.
 
Old thread, but I just wanted to say that I've tested it and that it's possible to set IMAP quota that way.
The email_change_pass_post.sh seems wrong to me with all the operations about newuser, newfile, etc., but if you remove all that, it works.

Thanks for testing it out and your positive feedback.

But what do you mean with "The email_change_pass_post.sh seems wrong". Can you explain what you think is wrong with it? In case the username of an imap account changes, the quota file has to be moved...
 
Can you explain what you think is wrong with it? In case the username of an imap account changes, the quota file has to be moved...
Not in my version of DA. If a user changes the name, DA just renames the folder and so nothing needs to be done to keep the quota file.

Other things that needed to be changed:
In dovecot_virtual_delivery, the user has to be the user or it's not possible to communicate with auth-master

In socket listen { master {, the mode has to be "0660" and the group set to "mail", otherwise, emails can't get transferred.

In the sh scripts, you need to have some extra lines that deal with users with unlimited quota.
 
Back
Top