Autoresponder problem

As I noticed that autoreponders did not function, I waited moving my websites to my DirectAdmin server. Since 1.195 and a new Exim, autoresponders should work fine (again). But……

There is another problem, which makes autoresponders still useless for me. A little explanation.

Autoresponders in DirectAdmin work fine (though I have some wishes, e.g. attachments). Even in combination with (one or more) forwarders etc. (the help information on e-mail is not up-to-date) it works great. But … only when a message comes from an e-mail program, e.g. Outlook.

When you have a form on your website (with a cgi-script, like Matts FormMail) and you want a responder on that as a confirmation to the sender, the responder does not go to the sender. It even doesn’t go to the recipient, it goes to the owner of the server….. addressed to [username]@[server]. Also the address of the sender is weird: [username]@[server] namens [[realname][ (email)] sender] (Namens is Dutch for On behalf). The header of the e-mail then looks like this (britishh is a username and websystent.nl is the name of my server):

Return-Path: <[email protected]>
Received: from websystent.nl (websystent.nl [213.247.46.205] (may be forged))
by reaktie9.reaktie.com (8.11.6/8.11.6) with ESMTP id h9LDg9a04775
for <[email protected]>; Tue, 21 Oct 2003 15:42:09 +0200
Received: from britishh by websystent.nl with local (Exim 4.20)
id 1ABwlu-0005B6-MO
for [email protected]; Tue, 21 Oct 2003 15:42:02 +0200
From: "W. van Logchem" <[email protected]>
To: [email protected]
Subject: Aanvraag Brochure W. van Logchem
Message-Id: <[email protected]>
Sender: <[email protected]>
Date: Tue, 21 Oct 2003 15:42:02 +0200
Status:

But should be something like this (send from Outlook 2000):

Return-Path: <[email protected]>
Received: from websystent.nl (websystent.nl [213.247.46.205] (may be forged))
by reaktie9.reaktie.com (8.11.6/8.11.6) with ESMTP id h9HLrKb16446
for <[email protected]>; Fri, 17 Oct 2003 23:53:20 +0200
Received: from [62.163.246.240] (helo=LeenP42)
by websystent.nl with asmtp (Exim 4.20)
id 1AAcX3-00067H-8C
for [email protected]; Fri, 17 Oct 2003 23:53:13 +0200
From: "Leen Lagendijk" <[email protected]>
To: <[email protected]>
Subject: Test
Date: Fri, 17 Oct 2003 23:52:57 +0200
Message-ID: <FB3F49B36F22D4119AA900805FEF68F30F3F01@consystent-nt4.consystent.nl>
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="----=_NextPart_000_0004_01C39509.CD5415B0"
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook, Build 10.0.4024
Importance: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
Status:

As I am not a UNIX or mail expert, I tried everything else. I used other CGI-script, I tried to change the sender and reply address in the CGI script (if possible), nothing helped. I did not try it in PHP because my PHP knowledge is too little, but I can imagine that the problem in PHP is the same. DirectAdmin (or Exim) seems to hate scripts in combination with responders. For safety reasons? I can’t think why.

As I have a lot of websites using this form/responder combination, even on different pages on one site, this feature makes me not happy.

Besides, on other webservers (Plesk, TurboLinux) from which I intend to move to DirectAdmin, I’ve got no problems with this. Also, as it is works fine in DirectAdmin with an e-mail send from Outlook, I think this must be a bug (in DirectAdmin or Exim).

I asked the hosting company where my server is co-located, for assistance and they tried to change the Exim config, also spending a couple of hours. They managed to get rid of the ‘namens‘, but not more than that. The responder keeps going to [username]@[server].

Fortunately this hosting company offers me space on their Plesk Servers for free for the concerning websites, until this problem is solved. I appreciate that very much, but then I have to move my websites twice. I don’t like that thought, therefore, until now, I keep these sites at my old hosting companies, paying twice.

Is there anybody who can help me?

Small addition to this story (Leen is a customer at our company)

The sites/users which apply to this, HAVE been added to the trusted users in the exim.conf, this should allow for fully customizable headers..

Adding britishh to the trusted user DOES change the returnpath to the postmaster entry which is set in the formmail script, but this isn't right, since this means the mail is bounced (while it isn't)

The problem is pretty complicated, and after a lot of searching on the internet, I've dug up a bunch of similair issues (try a search on: 'exim return-path' in google and google groups) unfortunately none with working solutions..


We're not too shabby on moving Leen over to Plesk, not only is the license for that worth a pot of gold, at every upgrade/update something breaks, eating away hours of my time fixing the stuff the upgrade broke..

So just to confirm what you're trying to do, you've got a form mail script which is owned and run by user "britishh" on the server and you want their email to go to you, and a confirmation to go to them.

The form script is the one that is generating the email, so unless you provide the appropriate headers, exim won't really know that it came from anyone else but the owner of the script.

You could try adding:
Reply-To: emailaddress
Return-Path: emailaddress

headers so that exim will know where it's from.. OR (this is what I would do it):

Have the form mail script email a normal email address like it does normally, but then also have the script generate another email message to be sent out to the person who wrote the message (no autoresponders anywhere).

Example, in php: So.. that line in addition to the regular mail will send out 2 emails to the proper people.

John

Well, actually the confirmation is off....

You type in the mailform as a random interested customer, including your mailadress.., this should be seen as the From: address..

The mailform forwards it to the autoresponder, and the autoresponder should jump in and reply to the address the customer filled in..

It works in a default qmail config like plesk uses, and it's not just a single mailform that could be hacked, there's another application Leen uses which needs it to act like that..

Initially it used the Sender: header to reply to (which is also [email protected] in this example), I managed to kill off that one with a headers_remove in exim.conf..

To no avail so far..

Hello,

The thing is, exim won't use the "From:" to send the reply, it will reply to "Reply-To:" or "Return-Path:" .... which are added by exim if you don't. Since your script probably isn't adding them, the mail is sent back to the sender of the original email.

John

Hi,

I just looked at my e-mail. Thanks John and Wunk!

Tomorrow (it is now bedtime in Holland) I'll try to change the text From: in the CGI script into Reply-To: or Return-Path: and see what happens then.

I let you know what the results are.

Thanks!

Hi John,

I used Matt's FormMail.pl and changed From: into Reply-To: and tested it. No success.

Then I tried with Return-Path and the last thing I tried was to put From:, Reply-To and Return-Path in the e-mail header. No success.

This is my last CGI-code in Matt's FormMail:

sub send_mail {
# Localize variables used in this subroutine. #
local($print_config,$key,$sort_order,$sorted_field,$env_report);

# Open The Mail Program
open(MAIL,"|$mailprog");

print MAIL "To: $Config{'recipient'}\n";
print MAIL "From: $Config{'email'} ($Config{'realname'})\n";
print MAIL "Reply-To: $Config{'email'} ($Config{'realname'})\n";
print MAIL "Return-Path: $Config{'email'} ($Config{'realname'})\n";

# Check for Message Subject
if ($Config{'subject'}) { print MAIL "Subject: $Config{'subject'}\n\n" }
else { print MAIL "Subject: WWW Form Submission\n\n" }

print MAIL "Below is the result of your feedback form. It was submitted by\n";
print MAIL "$Config{'realname'} ($Config{'email'}) on $date\n";
print MAIL "-" x 75 . "\n\n";

if (@Print_Config) {
foreach $print_config (@Print_Config) {
if ($Config{$print_config}) {
print MAIL "$print_config: $Config{$print_config}\n\n";
}
}
}

And these are the headers of two e-mails:

1.
Return-path: <[email protected]>
Envelope-to: [email protected]
Delivery-date: Fri, 07 Nov 2003 15:55:25 +0100
Received: from disc by websystent.nl with local (Exim 4.20)
id 1AI81E-0005qZ-SD
for [email protected]; Fri, 07 Nov 2003 15:55:24 +0100
To: [email protected]
Reply-To: [email protected] (Leen Lagendijk)
Subject: Test van discnl.com
Message-Id: <[email protected]>
From: [email protected]
Date: Fri, 07 Nov 2003 15:55:24 +0100

2.
Return-path: <[email protected]>
Envelope-to: [email protected]
Delivery-date: Fri, 07 Nov 2003 16:03:05 +0100
Received: from disc by websystent.nl with local (Exim 4.20)
id 1AI88f-0005ru-Dw
for [email protected]; Fri, 07 Nov 2003 16:03:05 +0100
To: [email protected]
Reply-To: [email protected] (Leen Lagendijk)
Subject: Test van discnl.com
Message-Id: <[email protected]>
From: [email protected]
Date: Fri, 07 Nov 2003 16:03:05 +0100

disc is the username of the account
websystent.nl is my servers name

The e-mail should go to [email protected] and the responder should go to [email protected].

Exim changes the Return-Path in |username|@|servername|, whatever I put in the CGI script.

I hope you have enough information?

Hello,

I guess exim is too smart for it's own good in this case

Looks like you'll just have to send 2 emails from the script, one for a confirmation from the user, and one for you.

John

That's it?

If exim is not able to find and use any of the headers you give it for the autoresponder, then I'm not sure what else there is.

John

I don't understand this.

As a simple 'website builder', I should wonder why an e-mail from Outlook gives a normal responder to the sender while a sender from a form can not receive a responder. That responder goes to the owner of the server.... How do I tell my customers?

Mmmm.... that's Exim... If you had chosen a webhoster using Plesk or TurboLinux there was no problem at all. But now you have chosen a webhoster using DirectAdmin and you have to change your CGI script.....

Please replace yourself in the skin of a customer with no experience of CGI, PHP etc.

To be honest, I already discussed with my supplier, Easyhosting, to convert my DirectAdmin server to Plesk because of this problem. It has costs me already so much time! First the responders did not work at all, and then the responders only work with Outlook…. Jeroen (Wunk) keeps me from doing this…. Why?

If it isn't DirectAdmin in which the problem can be solved and it is Exim, as part of DirectAdmin, as a supplier of Exim I should not like the idea that my customers are using a product, not ‘compatible’ with concurrent products!

Last week I succeeded in reprogramming my CGI script so that responders can be send from the script instead from DirectAdmin.

As I am not a Perl programmer it was a tough day for me, al those {/<[._(&)#$@!|]\;}….

I was very happy with the test results and I implemented the script at the site of one of my customers, a travelling agency. A reason to send responders is the confirmation for a reservation. I did not feel so happy when I received a ‘Failure notice’ from a mailer daemon addressed to username@servername ([email protected] in this case).

Still this problem in Exim! I know I can set up forwarders for my users, and if you don't have many responders that is OK.

I think this remains a very nasty problem and I really hope you can do something about it!

Hello,

Remember that you cannot use the same server name as domain that you have on the server. This will prevent any system emails from making it through (will be caught by the domain's catch-all instead of going to system email account).

Change your servername to something like : server.websystent.nl
and add an A record for "server" to the websystent.nl dns zone.

The autoresponder is currently set to stop processing if the autoresponder if found. If you want it to follow through to continue sending the mail, you could add an "unseen" line in the "userautoreply:" portion of the /etc/exim.conf so that exim will pretend it didn't notice an auto reply was sent out. That way the message will continue to be sent to a pop account if it exists.. but if it doesn't it will hit the catch-all (which may be set to :fail:, so be careful). OR you could use the CC option to forward the email to a different pop account.

John

Thank you for your answer! You mean I have chosen a wrong name for my server....?

Correct me if I'm misunderstanding. Any change of the server name implicates changes of the meserver names at the registrants?

I'll discus this with Jeroen.


Thanks!

Hello,

It's just for the hostname of the server. Changing it shouldn't affect anything (other than fix email). You don't need to change anything at your registrar.

Admin Panel -> Admin Settings -> Server Name : server.websystent.nl -> Save

Admin Panel -> DNS Admin -> websystent.nl
add an A record:
server -> 213.247.46.207

John

Might want to add that as a notice in the setupscript, all points in the direction to make the domainname the servername, since it'll make ns1 and ns2.servername.com when setting DA up using the scripts..

Since it's not common or wanted by customers to make a ns1 and ns2.servername.domainname.com

Hi John,

After renaming the server and adding the A record in the DNS the autoresponders and the failure messages disappear.... Of course, because there is no forwarder or email address for username@servername after the change (e.g. [email protected])

I decided to go back to the old situation, using my new script with the built-in responder and making a forwarder for my users on the server. This way failure messages are received on the right place and I can keep an eye on autoresponders that Exim don't want to send. I am very glad I don't have thousand users now!

With my new script the problem has become a minor issue, but with the failure messages it shows that Exim reacts different from other mail programs, e.g. from the Plesk.

I think this bug in Exim has to be reported and repaired.....

Exim and autoresponder problems

I agree that something has to be done about this problem of autoresponders not responding to email addresses sent by a form on the same server as the autoresponder email address.

I too can send an email from outlook and have the autoresponder behave just fine. As soon as any form based script sends an email to the same autoresponder address it fails.

This is definitely an exim problem and something that any of my hosting clients would expect as something that they would have working properly for them as part of my hosting services.

The advice about changing scripts to add another email is not the right answer since most hosting clients don't have the first clue about how to do custom editing of scripts. The only acceptable answer is to have exim autoresponders behave as expected and how its email competitors already have autoresonders behaving. This is not a DirectAdmin issue - it is an Exim limitation until Exim changes things in some future upgrade. Lobby Exim about this because there is nothing DirectAdmin can do about it.

The only other solution is to buy an autoresponder script that will work with Exim and set up a series of autoresponses that are triggered by the initial form submission. Example - a series of emails that might consist of a number of lessons on a specific subject that get emailed out every day or week or month for the duration of the course. These scripts work really nice and are a big step above any single event autoresponder found with Exim or sendmail

Thought I'd wake up this old thread in case anybody else has this problem.

We recently had a customer that was using a formmail on a remote server that was generating an email to an autoresponder on a DirectAdmin machine with us. The generated headers had an incorrect From: on the message, but the Reply-To was correct.

The fix on the autoresponder side was to adjust exim.conf's "userautoreply:" stanza a bit.

to = "${sender_address}"

change to:

to = "${reply_address}"

The difference between the two is below.

`$sender_address': When a message is being processed, this variable contains the sender's address that was received in the message's envelope.

`$reply_address': When a message is being processed, this variable contains the contents of the `Reply-to:' header if one exists, or otherwise the contents of the `From:' header.

Hope that helps somebody if they run into this in the future.

I see no reason not to make this default behavior in DA's exim.conf.

Thanks,

Eli Katzenmeyer

Where in DirectAdmin's exim.conf file?

I can't think of anywhere in exim.conf where using the reply address wouldn't result in opening up the server to use by spammers.

Perhaps you can enlighten me?

Jeff