Where all are the e-mail messages located?

IT_Architect

Verified User
Joined
Feb 27, 2006
Messages
1,114
I use Exim and Dovecot. In /home/<user> I see a Maildir and a imap directory. The imap directory seems to be the only one with messages. So can I assume that, other than outgoing messages, the imap structure contains all of the user's messages?

Thanks!
 
Last edited:
What I see on disk is:
- /home/$USER/Maildir/new is full of messages. The rest are empty.
- There are messages in:
/home/$USER/imap/$DOMAIN/$EMAILUSER/Maildir/cur
and
/home/$USER/imap/$DOMAIN/$EMAILUSER/Maildir/.INBOX.Sent/cur

So then:
- When an e-mail message arrives, all copies for the os user are stored in /home/$USER/Maildir/new?
- When an e-mail message arrives, all copies for the virtual e-mail user are stored in /home/$USER/imap/$DOMAIN/$EMAILUSER/Maildir/.INBOX.Sent/cur?
- When a client comes in POP3, he is getting a filtered messages from /home/$USER/Maildir/new, or /home/$USER/imap/$DOMAIN/$EMAILUSER/Maildir/.INBOX.Sent/cur?
- When you delete a message in webmail, are you also deleting it from POP mail? I'm not clear on the relationship between $USER/Maildir and imap/$DOMAIN/$EMAILUSER/Maildir.
 
@IT_Architect,

I'm not sure, what you are asking here. Is that how the things work on your server? Or it is how you want them to work?

Every IMAP folder (at least for incoming messages) has sub-folders new and cur. The new sub-folder contains all newly arrived messages which have not been yet viewed by any e-mail client (does not matter: offline or online). Of course you should read them via POP3 or IMAP, otherwise if you read them in SSH or FTP/FileManager they would not marked as read.

And cur sub-folder contains emails which were read by IMAP/POP3.

Note, by default all incoming emails are stored as a single copy when using IMAP.

Most Webmails can work both either by IMAP or POP3 protocol. And it depends on their settings whether an email will be deleted from IMAP folder or not on deleting it from webmail.

/home/$USER/Maildir/new is the system email box storage. It has nothing to do with virtual email boxes. In most cases the box contain errors and notifications from cron (if arrive any), and bounces.

So then:
- When an e-mail message arrives, all copies for the os user are stored in /home/$USER/Maildir/new?
- When an e-mail message arrives, all copies for the virtual e-mail user are stored in /home/$USER/imap/$DOMAIN/$EMAILUSER/Maildir/.INBOX.Sent/cur?
- When a client comes in POP3, he is getting a filtered messages from /home/$USER/Maildir/new, or /home/$USER/imap/$DOMAIN/$EMAILUSER/Maildir/.INBOX.Sent/cur?
- When you delete a message in webmail, are you also deleting it from POP mail? I'm not clear on the relationship between $USER/Maildir and imap/$DOMAIN/$EMAILUSER/Maildir.
 
I use Exim and Dovecot. In /home/<user> I see a Maildir and a imap directory. The imap directory seems to be the only one with messages. So can I assume that, other than outgoing messages, the imap structure contains all of the user's messages?
All virtual mailboxes, the ones you set up for users, are in /home/USERNAME/imap, the Maildir directory is for the main email accout (the same as the username). Whether or not the latter has any contents depends a lot on how you set up your site and your email.

Does that answer your question?

Jeff
 
I missed the OP's second post to the thread; my resplies are below:

When an e-mail message arrives, all copies for the os user are stored in /home/$USER/Maildir/new?
Messages are always stored in the new subdirectory until they're read; then they're moved to the cur directory. I've never seen a message in the tmp directory. The original specification was designed by Dan Bernstein, the author of Qmail. I've never seen documentation for use of the tmp directory; perhaps Dovecot and other IMAP servers use it on a transient basis while otherwise manipulating the files.
When an e-mail message arrives, all copies for the virtual e-mail user are stored in /home/$USER/imap/$DOMAIN/$EMAILUSER/Maildir/.INBOX.Sent/cur?
Same as above; stored in new until they're read; then moved to cur.
When a client comes in POP3, he is getting a filtered messages from /home/$USER/Maildir/new, or /home/$USER/imap/$DOMAIN/$EMAILUSER/Maildir/.INBOX.Sent/cur?
I'm not sure what you mean by filtered. Exactly what a client sees is dependent on the client software being used.

POP3 clients, if properly configured, should download all email not previously downloaded, from both cur and new and then notify Dovecot that they've downloaded the emails. At that point, all downloaded emails should be moved from new to cur, unless the POP3 client is set to delete all downloaded emails from the server, in which case they'll be deleted (and not saved anywhere on the server).
When you delete a message in webmail, are you also deleting it from POP mail?
While DirectAdmin often refers to mailboxes as POP3 email boxes, Dovecot makes no physical distinction at the mailbox level. Older DirectAdmin installations may still include the UebiMiau webmail client, often simply called webmail. At the time DirectAdmin was first developed it didn't unclude any IMAP functionality, and UebiMiau was the only generally recognized and accepted webmail client which could be configured to use POP3, so it was the only choice. (While UebiMiau is configurable to use IMAP it was never set up for IMAP in DirectAdmin).

The original POP3 server used by DirectAdmin didn't use the same email structure at all; it used the mbox structure, and a discussion of mbox is beyond the scope of this thread, so for the following let's presume only Dovecot is being used; Dovecot uses the same email subdirectory structure whether being accessed by POP3 or by IMAP.

UebiMiau as a POP client signals Dovecot that the new email has been downloaded, which causes it to move any email in the neew directory to the cur directory, and then UebiMiau displays the contents of the cur directory. With UebiMiau when you move an email to a different folder, the email will be moved to a proprietary subdirectory structure that only UebiMiau has access to, and neither other POP3 nor IMAP clients will be able to see it.

Other webmail clients use IMAP, and all IMAP clients view, retrieve, store, and otherwise manage email directly on the server. If you move an email to a different folder (subdirectory), you should see it there in any other IMAP client, whether on desktop, laptop, or even cellphone.
I'm not clear on the relationship between $USER/Maildir and imap/$DOMAIN/$EMAILUSER/Maildir.
As I've already written, the former is for the user who owns the domain, and the latter for virtual email accounts only.

I hope this now answers all the questions. If I've missed anything please bring it to my attention.

Thanks.

Jeff
 
- All virtual mailboxes, the ones you set up for users, are in /home/USERNAME/imap.
- The Maildir directory is for the main email accout (the same as the username)
- While DirectAdmin often refers to mailboxes as POP3 email boxes, Dovecot makes no physical distinction at the mailbox level.

- Every IMAP folder (at least for incoming messages) has sub-folders new and cur.
- The new sub-folder contains all newly arrived messages which have not been yet viewed by any e-mail client (does not matter: offline or online).
- The cur sub-folder contains emails which were read by IMAP/POP3.
- By default all incoming emails are stored as a single copy when using IMAP.
- /home/$USER/Maildir/new is the system email box storage. It has nothing to do with virtual email boxes. In most cases the box contain errors and notifications from cron (if arrive any), and bounces.
Got it! These replies help a lot to clarify how e-mail storage works.
- All received e-mail messages for virtual e-mail users are stored under IMAP, whether they are accessed via IMAP or POP3. Dovecot is handling both types of connections.
- Messages for the DA user are stored under the other maildir folder, and is normally cron messages etc.
- Messages initially go to the new directory until accessed by any mail client, IMAP or POP3, at which point they are moved to the cur directory.

Question: Under /home/<user>/imap/<domain>/<virtual user>/Maildir, there is a dovecot.index, etc. I had to restore someone's sent items last week. I chickened out and restored his entire /home/<user>/imap/<domain>/<virtual user>/Maildir because I figured it mess up dovecot and indexing. Is that logic good, or doesn't it matter if I were to restore just Sent Items?
Edit: I found the answer to this question: "If index files are missing, Dovecot creates them automatically when the mailbox is opened." Thus, the better way would have been to restore only the Sent Items folder, and then blow away the dovecot files in Under /home/<user>/imap/<domain>/<virtual user>/Maildir. Dovecot would have created a perfect new set files for the virtual user, and I would have lost no messages that had come in during the time between the customer's accidental deletion and my restore.

Other: This thread has been very helpful. I've been "winging it" all this time as to how e-mail worked. I got away with it until last week when I had to restore accidentally deleted e-mails, and this week when I found someone's e-mail password had been hacked, and I had to work with DA's output queue viewer, and Exim commands http://www.sant-media.co.uk/2011/03/exim-queue-management-tutorial/ to delete about 7 million messages from the output queue. The reasons I didn't get back to this thread earlier is I didn't get a notification until very late, and I was embroiled in several emergencies.

Thanks all!
 
Last edited:
Back
Top