Almost never ending mailqueue after migrating account

ericovk

ericovk

Verified User
Joined
Apr 17, 2012
Messages
228
Location
Rotterdam, Netherlands
Last friday I migrated an account from DA on a dedicated server to another DA account on a cluster environment. This account has put thousands of mails in the mailqueue. There are so many mails in the queue that DA is timing out if I try to view it inside DA.

When I try
Code: 
exim -bq
I get thousands of messages like these
Code: 
 3d  129K 1V324h-00061i-Q2 <>
          [email protected]

I also tried to reload the queue with
Code: 
exim -qff
(source: http://help.directadmin.com/item.php?id=112)
But then I also get a time out.

Edit: I don't know if I can safely empty the mailqueue without deleting the customers email messages?
 
Last edited:
The tools you find here (bradthemad.org) may help you figure out what's in the queue.

Once you know what's in the queue you can make some decisions as to what you should do with it.

Emptying it will mean the emails in it won't ever get delivered, but they likely won't anyway, or they wouldn't be so many in the queue.

So the first thing to do is probably pick one of the messages in the queue, search for it in the logs, and figure out why it ends up in the queue.

Often large queues indicate an account on the server sending tons of spam, so the server's outging email IP# gets blocked and mail gets refused, or a lot of spam sent to undelverable addresses ends up stuck in the queue.

Note you should check to see how many frozen messages are in the queue. They'll never get retried unless you unfreeze them, and they'll eventually get deleted. You can try delivering one more time and then deleting them, but again, you should know what they are and why they're there, first.

Last resort: delete the queue. You probably won't be able to do this during normal tools if it's too big. And rely on your TOS which hopefully tells your clients that email is not a guaranteed medium.

Jeff
 
3d 129K 1V324h-00061i-Q2 <>
[email protected]
Click to expand...
You might also use the mail queue manager from DA to read this email. This way you might be able to see from the headers which script is causing this email flow.
At least if you are using some extra settings in Exim.
 
Thanks for the info. The exim cheatsheet is very usefull.

It seems like all the messages contain the autoreply message from this customer.
The customers mailbox has also been exploded somehow. When the customer logs in on the webmail, she sees a database error. No messages can be viewed or fetched through Outlook or something.

The other email accounts on this server are doing well. Could it be something like that two out-of-office replies are replying into each other and somehow filling each others inboxes?
 
Is it possible to delete alle the messages from the mail queue specified by the sender? (from [email protected])
Click to expand...
Yes it is, but that won't fix anything, you have to search the cause of the problem.

Like I said you might be able to see which script is causing it, by adding extra parameters to your exim.conf.
Like this in exim.conf under LOG_SELECTER, be sure these are present:
Code: 
+connection_reject \
+address_rewrite \
+all_parents \
+arguments \

Next to that, you could set the logfile for mail send via php seperately.
In your php.ini change
Code: 
;mail.log =
to something like:
Code: 
mail.log = /var/log/phpmail.log
Don't forget the remove the ; in front of the line and restart apache after the change.

Also make sure this setting is set to on:
Code: 
mail.add_x_header = On
Which makes a notice of the originating script in the mail header. Very usefull.
 
Richard G said:
Code: 
+connection_reject \
+address_rewrite \
+all_parents \
Click to expand...

I've added the above to my testing copy of my latest SpamBlocker release candidate, just above the
Code: 
+arguments
line. My file now replaces log code at edit point #11 with:
Code: 
log_selector = \
  +delivery_size \
  +sender_on_delivery \
  +received_recipients \
  +received_sender \
  +smtp_confirmation \
  +subject \
  +smtp_incomplete_transaction \
  -dnslist_defer \
  -host_lookup_failed \
  -queue_run \
  -rejected_header \
  -retry_defer \
  -skip_delivery \
  +connection_reject \
  +address_rewrite \  
  +all_parents \
  +arguments
Do you think this will make the log files much bigger? Do you think I should use this as default for the next version?

Jeff
 
Hi Jeff.
I don't think it will make the log files much bigger. But from point of experience I could say my Exim now gave better headers and worked better with CSF/LFD.

I think I've got the same as you have now, but since I like everything nice and tidy, I always remove the bunch and replace it like this:
Code: 
log_selector = \
  +connection_reject \
  +address_rewrite \
  +all_parents \
  +arguments \
  +delivery_size \
  +sender_on_delivery \
  +received_recipients \
  +received_sender \
  +smtp_confirmation \
  +subject \
  +smtp_incomplete_transaction \
  -dnslist_defer \
  -host_lookup_failed \
  -queue_run \
  -rejected_header \
  -retry_defer \
  -skip_delivery
 
I'll try it for a while. I'm not sure what you mean by 'better headers' since this won't affect headers at all.

I'm also not sure why you believe yours is more 'tidy' than mine :).

When I create the new release I'll alphabetize them, first the - ones, and then the + ones.

Jeff
 
I'm not sure what you mean by 'better headers' since this won't affect headers at all.
Click to expand...
You're right, should be "better logs" or more extended logs.

Why mine is more tidy? Because mine has first all + and then all the - ones. Yours was + then - and then + again.:)

Since you say in the new release you will put them first the - ones and then the + ones.... does this make any difference in the working of Exim when putting the - ones first?
Because until now the + ones always were first as far as I can remember?

By the way... any clue why the php mail log file won't work, I don't see anything appearing in there yet.:(
 
@Richard and Jeff: Ok, I added the log_selector lines, but I couldn't find
mail.log
Click to expand...
or something with
add_x_header
Click to expand...
in the php.ini file. Can I just add this, or is this something bad?

@Peter: I created the Apache "server status page", but I can't see anything strange. Is there something I need to see?
 
Odd that you can't find those settings. The one from the mail.log is not a real problem, but this should really be in the php.ini:
; Add X-PHP-Originating-Script: that will include uid of the script followed by the filename
mail.add_x_header = On
Click to expand...
Which version of php are you using?
 
Looks you're still using an old php.ini.

I don't know how you can get a fresh one the easy way.
You can first backup your php.ini if you have made changes in there, then delete the original one.

After that you can set "php_ini=yes" in options.conf (if you are using custombuild).
After that I run:
Code: 
./build php n
from the custombuild directory.
That way php will be rebuild and you will also get a new php.ini, don't forget to change "php_ini=yes" back to php_ini=no in custombuild.
 
for CB 1.2+
Code: 
./build set php_ini yes
./build php_ini
./build set php_ini no
Normally works.... unless smtalk hasn't fixed the renaming of the defaut php.ini in 5.3+ tarbell
 
Last edited:
I have DirectAdmin 1.43.3. Maybe that has got something to do with it? (My hosting provider installed it a while ago on my VPS)

Edit:
When I do:
./build php_ini
Click to expand...

I get this message:
mail-header-patch=yes is set in the options.conf, but is not required with this php version.
To enable the X-Mail header, set mail.add_x_header to 1 in your php.ini
Click to expand...

It looks like I've been configuring something half?? :confused:
 
Last edited:
Strange, I have tried Peters way, but when that didn't gave the results I wanted I tried Richard's way. Both are refreshing the php.ini but none of them have anything like mail.log or add_x_header in them. (I search through it by opening with vi and then search with "/mail.log"). Is this strange or what?
 
I tried Richard's way
Click to expand...
Peter's way is the same way as my way, only his version is shorter.:)

mail-header-patch=yes is set in the options.conf, but is not required with this php version.
To enable the X-Mail header, set mail.add_x_header to 1 in your php.ini
Click to expand...
That's normal. You can set mail-header-patch=no in options.conf and that error is gone.

none of them have anything like mail.log or add_x_header in them.
Click to expand...
And you deleted the php.ini before doing the ./build php n?
Maybe you got more php.ini files. Normally it should be (in Centos/Fedora) in /usr/local/lib.
 
I tried so many things today, now I'm not sure anymore if I removed the php.ini before I updated php. I'll just try it again.
It seems like php.ini is in the right place:
$ whereis php.ini

php: /bin/php /usr/bin/php /etc/php.ini /etc/php.d /usr/local/bin/php /usr/local/lib/php.ini /usr/local/lib/php /usr/share/php /usr/share/man/man1/php.1.gz
Click to expand...

PS: I found out what the source of the endless mailqueue is. The customer set an out-of-office reply and the account has been sending thousands to itself. I don't know how this started in the first place.

Is there a way to maximize the number of out-of-office replies per recipient? And a way to delete all of the thousands out-of-office-replies with one command?
 
You must log in or register to reply here.
Back
Top