IMAP/Webmail Problems

R

roarkh

Verified User
Joined
Aug 30, 2005
Messages
139
Location
Bellingham, WA
I have a DirectAdmin box running on a 3ghz Pentium IV with 1gb of RAM (CentOS 4.1 OS).

When accessing accounts via IMAP the accounts will go on and off line intermittently. If I click "get mail" enough times it will always spring back to life eventually. Also, when accessing the same accounts via SquirrelMail about 50% of the time I get this error, "ERROR: Connection dropped by IMAP server." If I re-login a few times it will eventually make the connection properly and work as expected.

Top reports that CPU useage is never more than 15%, I have watched it for 15-20 minutes straight and usually it is at less than 3% so I don't think it has anything to do with the hardware being overtaxed.

There are a total of about 250 IMAP accounts on this server and it seems to be affecting all of them but never all at the same time.

Does anyone have any ideas why this may be happening? I have searched the forums and it seems that lots of people are having problems of one form or another with IMAP but not exactly this. I switched everyone from POP to IMAP because I was experiencing the duplicate email problem when people accessed their account through webmail but I have been told by some that accessing via pop has been a problem as well.

Thanks in advance.
 
Yes, I did read that thread. I have upgraded Apache but only to 1.3.34 (from 1.3.33), not 2.0 as you did. It looks like the upgrade to 1.3.34 must have upgraded php to 4.4.0 at the same time.

I'm not convinced that we're having the same problem however. You say that you can not get webmail to work at all. For me, it will work one minute just fine and then the next time I get the error. It seems random.

Also, I am getting timeouts connecting with email clients that are not web-based as well, throughout the day I would guess that my client says my mail server has become unavailable about 25% of the time. Other people get the same message at different times. Again, it seems somewhat random.

I read this thread as well...
http://directadmin.com/forum/showthread.php?s=&threadid=9424

I don't know if it applies or not, it seems to be mostly focused on the duplicate email problem and not the problem I am having.

Does anyone have any ideas why this may be happening?
 
I may have some more clues. It looks like the problems are worse for people that have larger email stores (<20mb or) though it does at times affect people with well under 1mb stores.

I just had a few people complain about the problem occuring again and I ssh'd into the computer and noticed that there is a zombie exim process ([exim] <defunct>). I know that there were no zombie processes when I checked this morning. Could that be symptom or cause of the problem?

I would really appreciate any input I can get, this is causing a great deal of grief for my users (and consequently for me as well!)

Thank you.
 
OK, thanks for the post. I just read the entire four pages of that thread again and do not see where the patch fixes anything except for the duplicate email problem. Am I missing something here? My problem is dropped connections at random intervals, I also have had the duplicate email problem but that is not the critical problem for me. Do you think that patching vm-pop3d will fix my dropped connection issues?

I also see that there also was a mention of downloading and installing imapback-2004c1.tar.gz, is this what may fix my problems. To be honest I am not sure what version of imapd is running on my server right now, it was installed fresh by DirectAdmin about 3 months ago and has not been updated since. The usr/local/directadmin/scripts/packages folder just contains an executable called imapd. Is there some way to find out what version that actually is?

Thanks again for your help.
 
Ok sorry.... yea i agree, it is a bit confusing...

Here's a pretty low down way to do it, and it works... ( if none of those exist you will need to create them )

cd /usr/src/misc

EMACS vm_pop3d.patch

Paste this information in as shown below, ignore the *****'s they are for spacing this file.

****************************************************

diff -urN da_vm-pop3d-1.1.7f.orig/extra.c da_vm-pop3d-1.1.7f/extra.c
--- da_vm-pop3d-1.1.7f.orig/extra.c 2005-01-17 11:13:17.000000000 -0700
+++ da_vm-pop3d-1.1.7f/extra.c 2005-09-03 10:20:52.000000000 -0700
@@ -255,6 +255,7 @@
unsigned int max_count = 0;
/* start with 1 because first line of mbox is From */
int empty_line = 1, line_count = 0, len = 0;
+ int imap_header = 0;

struct md5_ctx md5context;

@@ -282,6 +283,10 @@
while (strchr(buf, '\n') == NULL)
fgets(buf, FREAD_SIZE, mbox); /* get all of From line */

+ if(!imap_header && strstr(buf,"MAILER-DAEMON")) {
+ imap_header = 1; /* we have the imap header message */
+ }
+
if (num_messages > 0) { /* another message */
md5_finish_ctx(&md5context, messages[num_messages - 1].uidl);
}
@@ -322,8 +327,11 @@
}
/* creating md5 hash for the UIDL using just first 25 lines */
/* FiXMe: Should be only for header! -TEMHOTA */
- if (line_count++ < 26 && len > 0)
- md5_process_bytes(buf, len, &md5context);
+ if(strncmp(buf,"Status:",7) &&
+ strncmp(buf,"X-",2)) {
+ if (line_count++ < 26 && len > 0)
+ md5_process_bytes(buf, len, &md5context);
+ }
}

if (buf[0] == '\n')
@@ -335,6 +343,14 @@
if (num_messages > 0) {
md5_finish_ctx(&md5context, messages[num_messages - 1].uidl);
}
+ if(imap_header != 0) {
+ /* We have the imap header, lets always return
+ * this uidl for it so it won't download again.
+ * This message changes every time a message is
+ * read or deleted.
+ */
+ memset(messages[0].uidl,'\0',sizeof(messages[0].uidl));
+ }

return OK;
}

***************************************************
URL: files.directadmin.com/services/da_vm-pop3d-1.1.7f.tar.gz


Next save it and exit emacs

cd /
cd /usr/src/misc
wget http://files.directadmin.com/services/da_vm-pop3d-1.1.7f.tar.gz
tar xvzf da_vm-pop3d-1.1.7f.tar.gz
patch -p0 < vmpop3d.patch
cd da_vm-pop3d-1.1.7f
./configure
make
/etc/init.d/vm-pop3d stop
cp /usr/sbin/vm-pop3d /usr/sbin/vm-pop3d.orig
cp vm-pop3d /usr/sbin/vm-pop3d
/etc/init.d/vm-pop3d start


++++++++++++++++++++

Note the last two lines of the above config file require input so they may require you to manually put them in one by one.

Once this is done the system will work, it worked for me on 3 server just fine !

-rob
 
Wow, thank you again. That was above and beyond the call of duty. I will give it a go when I am back in the office and report back.
 
no problem at all ! i think this should take you about 5 - 10 minutes to accomplish at best, it's pretty simple ( once you got it all figured out )

let us know how that works for you, hopefully you can go back to POP ? lol
 
OK, I have installed the patch. It will be a while before I know how well it worked but I really appreciate your help. I still think you may have misunderstood exactly what I am trying to accomplish however. I really do not want to stop using imap, I would rather use imap for everyone. I'm not sure how patching my version of pop will stop my connection errors in Web Mail and my imap timeouts but we will see if it helps.

For anyone else following Rszkutak's directions I had to make one minor change near the end because it seems that the pop service would immediately restart after stopping it and I was unable to overwrite the file because it was in use.

So, after the 'make' command I would do the following:

mv /usr/sbin/vm-pop3d /usr/sbin/vm-pop3d.orig
cp vm-pop3d /usr/sbin/vm-pop3d
/etc/init.d/vm-pop3d restart

I will post again later after I have had some time to test.
 
Well so far, so good, at least as far as duplicate emails go. I set up a test account and sent five emails to it, checked it with POP (leave on server), checked it with squirrelmail and then checked it with POP again and did not get any duplicates.

Unfortunately since the patch caused the duplicate problem again this morning I have been dealing with some fallout from the users from that!

So far the dropped connection problem appears to not be happening but I need at least 24 hours of testing to be sure that this problem is fixed.

I also know that some people were occasionally getting duplicate emails even though they never used webmail, do you think that this patch will fix that as well?

Thank you for all of your help.
 
Hmmm I've been having this problem as well specifically for one customer. But you never know if he's just the only one that has called and complained.

There are so many people on my server with thousands of emails in their boxes that right now are tagged as unread. I can't imagine making them all go through the experience of having all of their emails tagged Unread. Yikes!

Is there any way that anyone can see to perform this update/patch without resetting the email status?

TIA ;)
 
Unfortunately since headers getting rewritten is the cause of the problem in the first place as I understand it, it is unlikely that there is another solution. For more information check out this very popular link...

http://www.directadmin.com/forum/showthread.php?s=&threadid=9424&highlight=imapd

On another note, the imap connection issues and SquirrelMail connection issues mentioned in my original post in this thread appear to still be happening. :-(
 
Well, after some testing, my problems are definitely not fixed. As long as I set people up with POP things appear to be OK but imap is still giving me fits and this is becoming an emergency situation for me now.

I am still getting the "ERROR: Connection dropped by IMAP server." message about 75% of the time when logging in to any of my accounts with SquirrelMail.

I have about 200 users configured with the Apple Mail program for imap set to check messages every 5 minutes. About 75% of the time the indicator icon for lost connection to mail server appears next to their account. About half of the users are on our LAN but it does not seem to make any difference if people are inside or outside our firewall.

Does anyone else have any ideas?

Thank you.
 
Do you have an instances configured in your /etc/xinetd.conf file? If you have the default CentOS configuration, then yours is most likely at 60. That means that you can have no more than 60 instances of a service running at any one time.

IMAP is one of those services that that remains open until the connection is broken, so if you have 250 IMAP users, you will easily burn through the number of allowable instances. POP3 on the other hand only keeps the connection open long enough to download all the messages, then disconnects. That is probably why POP3 is not giving you a problem.
 
Now that is a brilliant suggestion, I appreciate it so much. I changed instances from 60 to 250 and then typed...

/etc/init.d/xinetd restart

That should be all I need to do, right?

I will watch it for a bit and post back with results.
 
Well I am very happy to say that my email problems appears to have been fixed, it was the number of instances in /etc/xinetd.conf that was causing the connection problems.

I am extremely thankful to "toml" for his help and to "rszkutak" for all of his help as well. Not only is the connection problem taken care of but the duplicate email problem appears to be fixed as well.

I also want to drop a big thank you to John at DirectAdmin who went out of his way to help me as well. He mentioned in his last email that he has changed the install procedure for DirectAdmin to check xinetd.conf for 60 instances and up it to 256 during the install automatically so hopefully some grief will be saved for new CentOS users.
 
Having this problem as well:

User with a 43mb inbox would log in, after about 5 minutes get this:

Code: 
ERROR: Connection dropped by IMAP server.
Query: FETCH 1:* (FLAGS UID RFC822.SIZE INTERNALDATE BODY.PEEK[HEADER.FIELDS (Date To Cc From Subject X-Priority Importance Priority Content-Type)])

checked for defunct exims:

[root@zeus] # ps aux | grep exim
mail 6918 0.0 0.0 6628 1868 ? S May02 0:10 /usr/sbin/exim -bd -q15m -oP /var/run/exim.pid
root 30009 0.2 0.1 6636 2216 ? S 16:39 0:00 /usr/sbin/exim -Mc 1FbP4U-0007o0-U7
mail 30010 0.0 0.1 6640 2236 ? S 16:39 0:00 /usr/sbin/exim -Mc 1FbP4U-0007o0-U7
mail 30011 0.2 0.0 6636 1864 ? S 16:39 0:00 /usr/sbin/exim -oMr spam-scanned -bS
mail 30012 0.0 0.1 6640 2236 ? S 16:39 0:00 /usr/sbin/exim -Mc 1FbP4U-0007o0-U7
mail 30014 0.0 0.0 0 0 ? Z 16:39 0:00 [exim <defunct>]


so I'm not sure what's going on here.

van
 
You must log in or register to reply here.
Back
Top