Spamblock report

T

torp

Verified User
Joined
Jul 6, 2004
Messages
158
Location
Oslo
Here's a really simple script that scans your Exim logfiles to see how effective your spamblocking is. It's helpful for seeing which spamblocks are most effective, and therefore which you should run first.

I found it on WebHostingTalk (http://www.webhostingtalk.com/showthread.php?t=295983&page=2) - written by eMtnMan

I have modiied the original file slightly to reflect DA's Exim logfile structure. Simply put it on your server and run it.
 

Attachments

  • blockreport.sh.txt
    6.6 KB · Views: 604
Very nice clean simple data collection script. I modified it a bit , ran it , nice ;)

I saw a good expression from another poster on this forum this week: the KISS method, meaning : Keep It Simple Stupid.

I was very amused by that and it is so very true, i like the script.
 
Unfortunately, your attachment is not properly formatted (or I don't see it properly formatted)
So I can't just donwload and run it.

Could you copy and paste a formatted version of it ?
(I mean with CR at the end of each line of the script)

Thanks
 
Really simple stuff :)

Code: 
echo .......
echo
echo REJECTS:
tail -20 /var/log/exim/rejectlog
echo .......
echo
echo PANIC:
tail /var/log/exim/paniclog
printf "\n"
echo ------------------
echo ANTI-SPAM MONITOR
echo ------------------
grep -i "reject\|sender\|attachm\|virus\|no host\|no ip" /var/log/exim/mainlog > kilme
echo
printf " From: "
grep -m1 "" kilme | cut -d' ' -f1
printf " Thru: "
tail -1 kilme | cut -d' ' -f1
echo
echo ------------------
echo Deleted SPAM:
echo ------------------
printf "dsbl.org = "
grep -ci "dsbl.org" kilme
printf "spamcop = "
grep -ci "spamcop.net" kilme
printf "abuseat = "
grep -ci "abuseat.org" kilme
printf "spamhaus = "
grep -ci "spamhaus.org" kilme
printf "njabl.org = "
grep -ci "njabl.org" kilme
printf "SPAM Assn. = "
grep -ci "spam score" kilme
echo " ------"
printf "SPAM Kills = "
grep -ci "blackl\|spam score" kilme
echo
echo ------------------
echo Fake or Unsafe:
echo ------------------
printf "Virus/Atch = "
grep -ci "virus\|attachm" kilme
printf "Forged HELO= "
grep -ci "Forged HELO" kilme
printf "No Host = "
grep -ci "no host" kilme
printf "No IP = "
grep -ci "no ip" kilme
printf "No Relay = "
grep -ci "not permitted" kilme
printf "Bad Sender = "
grep -ci "sender" kilme
printf "Bad Recpt = "
grep -icv "Forged HELO\|no host\|no ip\|not permitted\|sender\|spam score\|blackl\|virus\|attachm" kilme
echo " ------"
printf "Fake Kills = "
grep -civ "spam score\|blackl" kilme
printf "\n"
echo ------------------
printf "TOTAL DELETE "
grep -c "" kilme
echo ------------------
echo
echo
echo
grep -i "reject\|sender\|attachm\|virus\|no host\|no ip" /var/log/exim/mainlog.1 > kilme
printf " From: "
grep -m1 "" kilme | cut -d' ' -f1
printf " Thru: "
tail -1 kilme | cut -d' ' -f1
echo
echo ------------------
echo Deleted SPAM:
echo ------------------
printf "dsbl.org = "
grep -ci "dsbl.org" kilme
printf "spamcop = "
grep -ci "spamcop.net" kilme
printf "abuseat = "
grep -ci "abuseat.org" kilme
printf "spamhaus = "
grep -ci "spamhaus.org" kilme
printf "njabl.org = "
grep -ci "njabl.org" kilme
printf "SPAM Assn. = "
grep -ci "spam score" kilme
echo " ------"
printf "SPAM Kills = "
grep -ci "blackl\|spam score" kilme
echo
echo ------------------
echo Fake or Unsafe:
echo ------------------
printf "Virus/Atch = "
grep -ci "virus\|attachm" kilme
printf "Forged HELO= "
grep -ci "Forged HELO" kilme
printf "No Host = "
grep -ci "no host" kilme
printf "No IP = "
grep -ci "no ip" kilme
printf "No Relay = "
grep -ci "not permitted" kilme
printf "Bad Sender = "
grep -ci "sender" kilme
printf "Bad Recpt = "
grep -icv "Forged HELO\|no host\|no ip\|not permitted\|sender\|spam score\|blackl\|virus\|attachm" kilme
echo " ------"
printf "Fake Kills = "
grep -civ "spam score\|blackl" kilme
printf "\n"
echo ------------------
printf "TOTAL DELETE "
grep -c "" kilme
echo ------------------
echo
echo
echo
echo
echo
grep -i "reject\|sender\|attachm\|virus\|no host\|no ip" /var/log/exim/mainlog.2 > kilme
printf " From: "
grep -m1 "" kilme | cut -d' ' -f1
printf " Thru: "
tail -1 kilme | cut -d' ' -f1
echo
echo ------------------
echo Deleted SPAM:
echo ------------------
printf "dsbl.org = "
grep -ci "dsbl.org" kilme
printf "spamcop = "
grep -ci "spamcop.net" kilme
printf "abuseat = "
grep -ci "abuseat.org" kilme
printf "spamhaus = "
grep -ci "spamhaus.org" kilme
printf "njabl.org = "
grep -ci "njabl.org" kilme
printf "SPAM Assn. = "
grep -ci "spam score" kilme
echo " ------"
printf "SPAM Kills = "
grep -ci "blackl\|spam score" kilme
echo
echo ------------------
echo Fake or Unsafe:
echo ------------------
printf "Virus/Atch = "
grep -ci "virus\|attachm" kilme
printf "Forged HELO= "
grep -ci "Forged HELO" kilme
printf "No Host = "
grep -ci "no host" kilme
printf "No IP = "
grep -ci "no ip" kilme
printf "No Relay = "
grep -ci "not permitted" kilme
printf "Bad Sender = "
grep -ci "sender" kilme
printf "Bad Recpt = "
grep -icv "Forged HELO\|no host\|no ip\|not permitted\|sender\|spam score\|blackl\|virus\|attachm" kilme
echo " ------"
printf "Fake Kills = "
grep -civ "spam score\|blackl" kilme
printf "\n"
echo ------------------
printf "TOTAL DELETE "
grep -c "" kilme
echo ------------------
echo
echo
echo
echo
grep -i "reject\|sender\|attachm\|virus\|no host\|no ip" /var/log/exim/mainlog.3 > kilme
printf " From: "
grep -m1 "" kilme | cut -d' ' -f1
printf " Thru: "
tail -1 kilme | cut -d' ' -f1
echo
echo ------------------
echo Deleted SPAM:
echo ------------------
printf "dsbl.org = "
grep -ci "dsbl.org" kilme
printf "spamcop = "
grep -ci "spamcop.net" kilme
printf "abuseat = "
grep -ci "abuseat.org" kilme
printf "spamhaus = "
grep -ci "spamhaus.org" kilme
printf "njabl.org = "
grep -ci "njabl.org" kilme
printf "SPAM Assn. = "
grep -ci "spam score" kilme
echo " ------"
printf "SPAM Kills = "
grep -ci "blackl\|spam score" kilme
echo
echo ------------------
echo Fake or Unsafe:
echo ------------------
printf "Virus/Atch = "
grep -ci "virus\|attachm" kilme
printf "Forged HELO= "
grep -ci "Forged HELO" kilme
printf "No Host = "
grep -ci "no host" kilme
printf "No IP = "
grep -ci "no ip" kilme
printf "No Relay = "
grep -ci "not permitted" kilme
printf "Bad Sender = "
grep -ci "sender" kilme
printf "Bad Recpt = "
grep -icv "Forged HELO\|no host\|no ip\|not permitted\|sender\|spam score\|blackl\|virus\|attachm" kilme
echo " ------"
printf "Fake Kills = "
grep -civ "spam score\|blackl" kilme
printf "\n"
echo ------------------
printf "TOTAL DELETE "
grep -c "" kilme
echo ------------------
echo
echo
echo
echo
echo
grep -i "reject\|sender\|attachm\|virus\|no host\|no ip" /var/log/exim/mainlog.4 > kilme
printf " From: "
grep -m1 "" kilme | cut -d' ' -f1
printf " Thru: "
tail -1 kilme | cut -d' ' -f1
echo
echo ------------------
echo Deleted SPAM:
echo ------------------
printf "dsbl.org = "
grep -ci "dsbl.org" kilme
printf "spamcop = "
grep -ci "spamcop.net" kilme
printf "abuseat = "
grep -ci "abuseat.org" kilme
printf "spamhaus = "
grep -ci "spamhaus.org" kilme
printf "njabl.org = "
grep -ci "njabl.org" kilme
printf "SPAM Assn. = "
grep -ci "spam score" kilme
echo " ------"
printf "SPAM Kills = "
grep -ci "blackl\|spam score" kilme
echo
echo ------------------
echo Fake or Unsafe:
echo ------------------
printf "Virus/Atch = "
grep -ci "virus\|attachm" kilme
printf "Forged HELO= "
grep -ci "Forged HELO" kilme
printf "No Host = "
grep -ci "no host" kilme
printf "No IP = "
grep -ci "no ip" kilme
printf "No Relay = "
grep -ci "not permitted" kilme
printf "Bad Sender = "
grep -ci "sender" kilme
printf "Bad Recpt = "
grep -icv "Forged HELO\|no host\|no ip\|not permitted\|sender\|spam score\|blackl\|virus\|attachm" kilme
echo " ------"
printf "Fake Kills = "
grep -civ "spam score\|blackl" kilme
printf "\n"
echo ------------------
printf "TOTAL DELETE "
grep -c "" kilme
echo ------------
 
Last edited:
Forgive my ignorance, in what folder would I place this script on the server to run it?

Thanks for the help.
 
rocketcity said:
Forgive my ignorance, in what folder would I place this script on the server to run it?

Thanks for the help.
Click to expand...
Anywhere - and make executable
 
I always use chmod +x <filename.sh> , i also call these shell script files .sh so i can defer it with other scripts. Just like you would name a perl script .pl

I find it handy to place these things in /usr/local/bin for example.
 
torp & vingertop:

Thanks for the help. I now have it setup and working. After some modification, I have the script showing the number of attempted emails and the number of completed emails. I also setup a cron job that run's the script and emails it to me once a day with the results.

For anyone interested: I modified the following line:
Code: 
grep -i "reject\|sender\|attachm\|virus\|no host\|no ip" /var/log/exim/mainlog > kilme
to this:
Code: 
grep -i "reject\|sender\|attachm\|virus\|no host\|no ip\|completed\|H=" /var/log/exim/mainlog > kilme
Then modified the script to look like this:
Code: 
echo ---------------------------------
echo Email Monitor MainLog
echo ---------------------------------
grep -i "reject\|sender\|attachm\|virus\|no host\|no ip\|completed\|H=" /var/log/exim/mainlog > kilme
echo
printf " From: "
grep -m1 "" kilme | cut -d' ' -f1
printf " Thru: "
tail -1 kilme | cut -d' ' -f1
echo
echo
echo ----------------------------------
echo Total Attempted Emails
echo ----------------------------------
printf "Attempted Emails = "
grep -ci "H=" kilme
echo
printf "Completed Received Email = "
grep -ci "Completed" kilme
echo
I modified the remainder of the script to allow the same view for the previous log files.
 
sbreport.sh

Here's a silly script I wrote tonight that should hopefully display the current number of emails blocked by SpamBlocker in the exim rejectlog. The output should be easy to integrate into a webpage.

Code: 
#!/bin/sh
#
# sbreport.sh - simple script to display number of emails blocked by SpamBlocker
# [c]clint anderson - 2006/02/17 - script distributed under the BSD license

RBL_LIST=`grep "deny message" /etc/exim.conf|cut -f 2 -d "="|cut -f 1 -d "-"|cut -c 2-|sort|uniq`

IFS=$'\n'

for RBL in ${RBL_LIST}
do
    echo "${RBL}= `grep -ci ${RBL} /var/log/exim/rejectlog`"
done

exit

The output shoud look like this:

Code: 
# ./sbreport.sh
Email blocked by BSAL = 0
Email blocked by BSHL = 0
Email blocked by CBL = 15
Email blocked by LBL = 0
Email blocked by NJABL = 2
Email blocked by ORDB = 0
Email blocked by SORBS = 4
Email blocked by SPAMCOP = 19
Email blocked by SPAMHAUS = 0

Changed script to look at rejectlog instead of mainlog based on Jeff's suggestion. It's 5x faster now.
 
Last edited:
Unless you're running your own log formats you can run this against rejectlog rather than mainlog, and it'll take less time to run.

All emails rejected by ACLs should be in rejectlog as well as in mainlog.

Don't forget that once an email is blocked it's blocked, so changing the order exim.conf uses to check blocklists may significantly alter your results without affecting the overall final result at all.

Jeff
 
Putting it on a webpage

Sorry for such a dumb request.

I'd like to be able to call these scripts from a webpage and present it to anybody who wants to see it.

I am sure somewhere somebody has explained how to do this, but I can't find it.
 
Use the following...........


Code: 
#!/bin/sh
#
# sbreport.sh - simple script to display number of emails blocked by SpamBlocker
# [c]clint anderson - 2006/02/17 - script distributed under the BSD license
# HTML formatting added by Onno Vrijburg 2006/03/18
# To use change the webpage variable below to point to where you want the file created
# Create a header.php and footer.php file in the same directory as the file you are having the script create.

webpage="/path/to/your/page.php"

RBL_LIST=`grep "deny message" /etc/exim.conf|cut -f 2 -d "="|cut -f 1 -d "-"|cut -c 2-|sort|uniq`

IFS=$'\n'

echo "" > $webpage
echo "<?php" >> $webpage
echo " require_once('header.php');" >> $webpage
echo "?> " >> $webpage
echo "<table border='1' cellspacing='0' cellpadding='0'>" >> $webpage
echo "  <caption>" >> $webpage
echo "  Number of emails blocked by SpamBlocker" >> $webpage
echo "  </caption>" >> $webpage
echo "  <tr>" >> $webpage
echo "    <th scope='col'>Blocklist</th>" >> $webpage
echo "    <th scope='col'># blocked</th>" >> $webpage
echo "  </tr>" >> $webpage

for RBL in ${RBL_LIST}
do
        echo "  <tr>" >> $webpage
                echo "    <td>${RBL}</td>" >> $webpage
                echo "    <td>`grep -ci ${RBL} /var/log/exim/rejectlog`</td>" >> $webpage
                echo "  </tr>" >> $webpage
done

echo "</table>" >> $webpage
echo " " >> $webpage
echo " " >> $webpage
echo "<?php" >> $webpage
echo " require_once('footer.php');" >> $webpage
echo "?>" >> $webpage

exit

Be sure to create the header.php and foorter.php and to specify the path to your file....
 
Thank you Onno, this will help a lot of people.

I already solved it by modifying the previous script a bit to generate a list of the RBLs I am using, plus a total rejects and total accepted. Then I output the result to a file on the domain that I am using, www.mataspam.com,. (kill spam in Spanish). Then I use a SSI include to pull up the file into the webpage. The modified script runs on a cron job every 15 minutes to show people the current results of the anti spam scan. Probably not the most elegant way, but then, I am just a hack. ;)

You can see the results there (spanish site). They may not seem large, but then, I do not have a lot of customers yet.

I hope to be able to eventually make it into a database with graphs, etc.

My goal is to accomplish a very localized and aggresive Panama anti spam service for select customers.

Panamanian email brokers have created these huge lists of local emails, and users don't know how to report them, therefore they are hardly ever listed in the RBLs.

When I get another server, I hope I will be able to create a Panama specific rbl dns service. I HATE SPAM. :)
 
You must log in or register to reply here.
Back
Top