ipfw shows ip being blocked in dynamic rules

mart_nl

Verified User
Joined
May 31, 2012
Messages
48
Location
The Netherlands
Hi,

One of my customers cannot connect to his website on port 80. But is able to connect to port 2222 and also able to send mail to the server.

More details on the problem in my previous thread:
http://www.directadmin.com/forum/showthread.php?t=44547&page=1

We couldn't quite figure it out, and solved the problem by changing his MAC Address in his router, he then received a different IP-address and was able to visit the site.

However, after a few days ... same problem.

Now I've listed the dynamic rules of IPFW and found his IP multiple times there. It seems IPFW is blocking his IP on port 80 actively.

Anyone able to explain to me why, what, where because the rules in ipfw.rules are exactly the same as this thread:
http://www.directadmin.com/forum/showthread.php?t=42202&page=1

And this is the only person having troubles.

ipfw -d list shows:

60007 2429 106298 (133s) LIMIT tcp x.x.x.60 56311 <-> x.x.x.145 80
60007 2429 106297 (133s) LIMIT tcp x.x.x.60 56305 <-> x.x.x.145 80
60007 2435 110424 (133s) LIMIT tcp x.x.x.60 56304 <-> x.x.x.145 80
60007 2433 109073 (133s) LIMIT tcp x.x.x.60 56307 <-> x.x.x.145 80
60007 2437 111869 (133s) LIMIT tcp x.x.x.60 56295 <-> x.x.x.145 80
60007 2446 120421 (133s) LIMIT tcp x.x.x.60 56294 <-> x.x.x.145 80

These are just a few of the times this ip is listed in the ipfw -d list.

I hope to find the reason why it seems to be limiting his ip to my server ip on port 80. And find a way to stop this ...

Any help appreciated !

Kind regards,
Martin
 

zEitEr

Super Moderator
Joined
Apr 11, 2005
Messages
13,902
Location
GMT +7.00
Hello,

He gets blocked because of this rule

Code:
$cmd 60007 allow tcp from any to any 80 setup limit src-addr 20
You might need to increase 20 to something bigger, or investigate on why he sends so many connection requests to your server.
 

mart_nl

Verified User
Joined
May 31, 2012
Messages
48
Location
The Netherlands
Hello,

He gets blocked because of this rule

Code:
$cmd 60007 allow tcp from any to any 80 setup limit src-addr 20
You might need to increase 20 to something bigger, or investigate on why he sends so many connection requests to your server.
A very very nice catch there. Thank you !

Setup of the website is Joomla 2.5 with VirtueMart 2. I'm suspecting some plugin of Virtuemart or VirtueMart itself is responsible for this. Might be some bug in scripting that loops and throws him out.

We need to investigate this and thanks to this I've got a lead.

However, is there anyway to tell IPFW to 'reset' itself and allow the customer to continue working and record date, time and action last taken at the moment he will be blocked yet again which is obvious to happen sooner or later.

Thanks,
Martin
 

zEitEr

Super Moderator
Joined
Apr 11, 2005
Messages
13,902
Location
GMT +7.00
Not sure I understand what you mean right here:

However, is there anyway to tell IPFW to 'reset' itself and allow the customer to continue working and record date, time and action last taken at the moment he will be blocked yet again which is obvious to happen sooner or later.
Do you want just log the action without blocking or what? Note, all dynamic rules have a limited lifetime.
 

mart_nl

Verified User
Joined
May 31, 2012
Messages
48
Location
The Netherlands
Not sure I understand what you mean right here:



Do you want just log the action without blocking or what? Note, all dynamic rules have a limited lifetime.
I found out rebooting the server would clear the dynamic rules. However, today I learned some else is experiencing problems too. This person is unable to access FTP. The symptoms are simular to the problem of this topic.

Besides changing the ipfw.rules to allow for higher scores (it seems like it's a bit too tight) I was looking for a way to clear the dynamic rules using another method than rebooting the server.

I'm not running a commercial hosting business so it's not a too big a problem to reboot the server, but this just doesn't seem like the best way to 'clear' any unwanted blocks.

The person having troubles accessing FTP right now was uploading a WordPress website. So I'm trying to find out the best way to tune my ipfw.rules to be sure to prevent unwanted blocks while keeping things safe.

Current ipfw.rules

Code:
#!/bin/sh
#################################################
# ipfw Firewall Commands
#################################################
cmd="ipfw -q add"
ipfw -q -f flush
pif="vtnet0"

#################################################
# Allow Loopback and Deny Loopback Spoofing
#################################################
$cmd allow all from any to any via lo0
$cmd deny all from any to 127.0.0.0/8
$cmd deny all from 127.0.0.0/8 to any
$cmd deny tcp from any to any frag

#################################################
# Stateful rules
#################################################
$cmd check-state
$cmd deny tcp from any to any established
$cmd allow all from any to any out keep-state
$cmd allow icmp from any to any

#################################################
# Table 10 for IP blocks
#################################################
ipfw -q table 10 add 127.0.0.2
ipfw -q add 900 deny ip from 'table(10)' to any

#################################################
# Incoming/Outgoing Services
#################################################
$cmd 60001 allow tcp from any to any 21 setup limit src-addr 10
$cmd 60002 allow tcp from any to any 22 setup limit src-addr 8
$cmd 60003 allow tcp from any to any 25 setup limit src-addr 10
$cmd 60004 allow tcp from any to any 587 setup limit src-addr 20
$cmd 60005 allow tcp from any to any 53 setup limit src-addr 3
$cmd 60006 allow udp from any to any 53 limit src-addr 3
$cmd 60007 allow tcp from any to any 80 setup limit src-addr 20
$cmd 60008 allow tcp from any to any 110 setup limit src-addr 20
$cmd 60009 allow tcp from any to any 143 setup limit src-addr 10
$cmd 60010 allow tcp from any to any 443 setup limit src-addr 10
$cmd 60011 allow tcp from any to any 2222 setup limit src-addr 12
$cmd 60012 allow tcp from any to any 35000-35999 in setup limit src-addr 10
$cmd 60013 allow tcp from any to any 993 setup limit src-addr 10
$cmd 60014 allow tcp from any to any 995 setup limit src-addr 10
$cmd 60015 allow tcp from any to any 465 setup limit src-addr 10
$cmd 60016 allow tcp from any to any 585 setup limit src-addr 10


#################################################
# Deny Port scanning (Nmap)
#################################################
$cmd 00600 deny log logamount 50 ip from any to any ipoptions rr
$cmd 00610 deny log logamount 50 ip from any to any ipoptions ts
$cmd 00620 deny log logamount 50 ip from any to any ipoptions lsrr
$cmd 00630 deny log logamount 50 ip from any to any ipoptions ssrr
$cmd 00640 deny log logamount 50 tcp from any to any tcpflags syn,fin
$cmd 00650 deny log logamount 50 tcp from any to any tcpflags syn,rst


#################################################
# Deny and Log
#################################################
$cmd deny log all from any to any
 

mart_nl

Verified User
Joined
May 31, 2012
Messages
48
Location
The Netherlands
Just run

Code:
sh /etc/ipfw.rules
to reset (reload) your rules.
Just noticed this in my security log:

Code:
Oct  4 17:43:48 hosting kernel: ipfw: 60001 drop session x.x.x.211:50472 -> x.x.x.145:21, too many entries
So indeed ipfw.rules was blocking this person. Seems like this is something with FileZilla. My previous hosting provider kept blocking me from time to time whenever I would upload Joomla sites containing many (smaller) files.

Your suggested solution to clear the dynamic rules is to run the rules file which makes sense, I've tried this previously:

Code:
ipfw -q -f flush
It's about the same. The result was that I couldn't get my console back. It took forever to finish and finally I rebooted my server.

I thought maybe your solution would be a bit more graceful, however the result is the same.

The server is taking a long time to finish. I'm giving it some more time and hope it will finish.

(server is responsive though, ftp is up and sites are running)
 

mart_nl

Verified User
Joined
May 31, 2012
Messages
48
Location
The Netherlands
Code:
# sh /etc/ipfw.rules
Read from remote host xyz.nl: Operation timed out
Connection to xyz.nl closed.
After about 5 minutes this message appeared. I logged in again and checked the dynamic rules. Seems like it worked, it just took a while.
 

zEitEr

Super Moderator
Joined
Apr 11, 2005
Messages
13,902
Location
GMT +7.00
What do you see with

Code:
ipfw show 65535
?

Note, if you've got default rule in ipfw to block everything, then with running

Code:
ipfw -q -f flush
you'll lost connection to the server, and you'll need to reboot your server.

With restarting ipfw rules, you're more secured.
 

zEitEr

Super Moderator
Joined
Apr 11, 2005
Messages
13,902
Location
GMT +7.00
That is what I was writing about, if you run

Code:
ipfw -q -f flush
then the only rule which will be left will be the default rule

Code:
65535 1 64 deny ip from any to any
So you should be very careful when you reset/restart rules... You might want to switch default rule to allow all.
 

mart_nl

Verified User
Joined
May 31, 2012
Messages
48
Location
The Netherlands
Code:
65535 1 64 deny ip from any to any
So you should be very careful when you reset/restart rules... You might want to switch default rule to allow all.
I see your point. Can you elaborate on that ? From what I can find on best practices it envolves build kernel to IPFIREWALL_DEFAULT_TO_ACCEPT. I want to make sure to fix this the right way in a DirectAdmin environment.

Also, I'm wondering if it is at all possible to edit the ipfw.rules to default allow a particular ip-address so it will never be blocked (or has higher counters) as opposed to just incrementing the counters to a (much) higher value.
 

zEitEr

Super Moderator
Joined
Apr 11, 2005
Messages
13,902
Location
GMT +7.00
I want to make sure to fix this the right way in a DirectAdmin environment.
Directadmin as far as I know does not regulate this aspect of using firewall. It's just up to administrator of the server.

As you've got missing numbering from 900 and 60001, you can add there any rules you might want or need, for example:

Code:
# Allow all connections from YOUR-IP to the server
$cmd 1000 allow tcp from YOUR-IP to me
Replace YOUR-IP with actual IP from which you connect to your server, here you can add several comma separated IPs, and use netmasks.

If you want to allow more connection for trusted IPs, then add these lines:

Code:
$cmd -f table 2 flush
#################################################
# Add trusted IP and subnets into table 2
#################################################
$cmd table 2 add 1.2.3.4
$cmd table 2 add 11.22.33.45/24
$cmd table 2 add 11.22.44.40/24
$cmd table 2 add 13.13.13.0/23
#################################################
# you can add as many IPs as you need here...
#################################################


#################################################
# ALLOW MORE CONNECTIONS FOR TRUSTED IPS
# I MULTIPLIED THEM BY 3 (FROM ORIGINAL)
# YOU CAN CHANGE THEM AS YOU WISH TO FILL YOUR NEEDS
#################################################
$cmd 10001 allow tcp from 'table(2)' to me 21 setup limit src-addr 30
$cmd 10002 allow tcp from 'table(2)' to me 22 setup limit src-addr 24
$cmd 10003 allow tcp from 'table(2)' to me 25 setup limit src-addr 30
$cmd 10004 allow tcp from 'table(2)' to me 587 setup limit src-addr 60
$cmd 10005 allow tcp from 'table(2)' to me 53 setup limit src-addr 9
$cmd 10006 allow udp from 'table(2)' to me 53 limit src-addr 9
$cmd 10007 allow tcp from 'table(2)' to me 80 setup limit src-addr 60
$cmd 10008 allow tcp from 'table(2)' to me 110 setup limit src-addr 60
$cmd 10009 allow tcp from 'table(2)' to me 143 setup limit src-addr 30
$cmd 10010 allow tcp from 'table(2)' to me 443 setup limit src-addr 30
$cmd 10011 allow tcp from 'table(2)' to me 2222 setup limit src-addr 36
$cmd 10012 allow tcp from 'table(2)' to me 35000-35999 in setup limit src-addr 30
$cmd 10013 allow tcp from 'table(2)' to me 993 setup limit src-addr 30
$cmd 10014 allow tcp from 'table(2)' to me 995 setup limit src-addr 30
$cmd 10015 allow tcp from 'table(2)' to me 465 setup limit src-addr 30
$cmd 10016 allow tcp from 'table(2)' to me 585 setup limit src-addr 30
right after

Code:
ipfw -q add 900 deny ip from 'table(10)' to any
and before

Code:
#################################################
# Incoming/Outgoing Services
#################################################
$cmd 60001 allow tcp from any to any 21 setup limit src-addr 10

For further information on how to use ipfw you should read docs and manuals.
 

mart_nl

Verified User
Joined
May 31, 2012
Messages
48
Location
The Netherlands
Hi Alex,

While figuring out where to put the statue I intend to build in your honor, I edited my ipfw.rules according to your comments.

Thanks!
Martin
 

mart_nl

Verified User
Joined
May 31, 2012
Messages
48
Location
The Netherlands
It seems ipfw is ignoring this change to ipfw.rules.

I've added:

Code:
$cmd table 2 add 62.x.x.x
right after

Code:
#################################################
# Table 10 for IP blocks
#################################################
ipfw -q table 10 add 127.0.0.2
ipfw -q add 900 deny ip from 'table(10)' to any
and

Code:
$cmd 10007 allow tcp from 'table(2)' to any 80 setup limit src-addr 80
(also tried 'to me' instead of 'to any')

Yet the ip 62.* is still being blocked but not because of rule 10007 which you would expect it to do, but because of this:

Code:
Oct 30 19:13:32 hosting kernel: ipfw: 60007 drop session 62.x.x.x:59380 -> 141.x.x.x:80, too many entries
Seems like it is ignoring the rule. The customer is currently updating it's site and I have unblock him at daily base.

Pulling my hairs why it's ignoring the new ipfw.rules changes ... Any idea ?

Thanks,
Martin
 

mart_nl

Verified User
Joined
May 31, 2012
Messages
48
Location
The Netherlands
sure

Code:
#!/bin/sh
#################################################
# ipfw Firewall Commands
#################################################
cmd="ipfw -q add"
ipfw -q -f flush
pif="vtnet0"

#################################################
# Allow Loopback and Deny Loopback Spoofing
#################################################
$cmd allow all from any to any via lo0
$cmd deny all from any to 127.0.0.0/8
$cmd deny all from 127.0.0.0/8 to any
$cmd deny tcp from any to any frag

#################################################
# Stateful rules
#################################################
$cmd check-state
$cmd deny tcp from any to any established
$cmd allow all from any to any out keep-state
$cmd allow icmp from any to any

#################################################
# Table 10 for IP blocks
#################################################
ipfw -q table 10 add 127.0.0.2
ipfw -q add 900 deny ip from 'table(10)' to any

# 2012-10-07 <---
#################################################
# Add trusted IP and subnets into table 2
#################################################
$cmd table 2 add 80.x.x.x
$cmd table 2 add 81.x.x.x
$cmd table 2 add 66.x.x.x
$cmd table 2 add 62.x.x.x
#################################################
# you can add as many IPs as you need here...
#################################################

#################################################
# ALLOW MORE CONNECTIONS FOR TRUSTED IPS
# YOU CAN CHANGE THEM AS YOU WISH TO FILL YOUR NEEDS
#################################################
$cmd 10001 allow tcp from 'table(2)' to any 21 setup limit src-addr 30
$cmd 10002 allow tcp from 'table(2)' to any 22 setup limit src-addr 24
$cmd 10003 allow tcp from 'table(2)' to any 25 setup limit src-addr 30
$cmd 10004 allow tcp from 'table(2)' to any 587 setup limit src-addr 60
$cmd 10005 allow tcp from 'table(2)' to any 53 setup limit src-addr 9
$cmd 10006 allow udp from 'table(2)' to any 53 limit src-addr 9
$cmd 10007 allow tcp from 'table(2)' to any 80 setup limit src-addr 80
$cmd 10008 allow tcp from 'table(2)' to any 110 setup limit src-addr 60
$cmd 10009 allow tcp from 'table(2)' to any 143 setup limit src-addr 30
$cmd 10010 allow tcp from 'table(2)' to any 443 setup limit src-addr 30
$cmd 10011 allow tcp from 'table(2)' to any 2222 setup limit src-addr 36
$cmd 10012 allow tcp from 'table(2)' to any 35000-35999 in setup limit src-addr 30
$cmd 10013 allow tcp from 'table(2)' to any 993 setup limit src-addr 30
$cmd 10014 allow tcp from 'table(2)' to any 995 setup limit src-addr 30
$cmd 10015 allow tcp from 'table(2)' to any 465 setup limit src-addr 30
$cmd 10016 allow tcp from 'table(2)' to any 585 setup limit src-addr 30

# --> 2012-10-07

#################################################
# Incoming/Outgoing Services // 2012-10-07 values for 21/22/80 * 2
#################################################
$cmd 60001 allow tcp from any to any 21 setup limit src-addr 20
$cmd 60002 allow tcp from any to any 22 setup limit src-addr 16
$cmd 60003 allow tcp from any to any 25 setup limit src-addr 10
$cmd 60004 allow tcp from any to any 587 setup limit src-addr 20
$cmd 60005 allow tcp from any to any 53 setup limit src-addr 3
$cmd 60006 allow udp from any to any 53 limit src-addr 3
$cmd 60007 allow tcp from any to any 80 setup limit src-addr 40
$cmd 60008 allow tcp from any to any 110 setup limit src-addr 20
$cmd 60009 allow tcp from any to any 143 setup limit src-addr 10
$cmd 60010 allow tcp from any to any 443 setup limit src-addr 10
$cmd 60011 allow tcp from any to any 2222 setup limit src-addr 12
$cmd 60012 allow tcp from any to any 35000-35999 in setup limit src-addr 10
$cmd 60013 allow tcp from any to any 993 setup limit src-addr 10
$cmd 60014 allow tcp from any to any 995 setup limit src-addr 10
$cmd 60015 allow tcp from any to any 465 setup limit src-addr 10
$cmd 60016 allow tcp from any to any 585 setup limit src-addr 10


#################################################
# Deny Port scanning (Nmap)
#################################################
$cmd 00600 deny log logamount 50 ip from any to any ipoptions rr
$cmd 00610 deny log logamount 50 ip from any to any ipoptions ts
$cmd 00620 deny log logamount 50 ip from any to any ipoptions lsrr
$cmd 00630 deny log logamount 50 ip from any to any ipoptions ssrr
$cmd 00640 deny log logamount 50 tcp from any to any tcpflags syn,fin
$cmd 00650 deny log logamount 50 tcp from any to any tcpflags syn,rst


#################################################
# Deny and Log
#################################################
$cmd deny log all from any to any
 

zEitEr

Super Moderator
Joined
Apr 11, 2005
Messages
13,902
Location
GMT +7.00
Sorry, I was meaning a list of running rules, which you can retrieve with

Code:
ipfw show
in would include counters.
 

mart_nl

Verified User
Joined
May 31, 2012
Messages
48
Location
The Netherlands
Code:
# ipfw show
00100   8124    843548 allow ip from any to any via lo0
00200      0         0 deny ip from any to 127.0.0.0/8
00300      0         0 deny ip from 127.0.0.0/8 to any
00400      0         0 deny tcp from any to any frag
00500      0         0 check-state
00600   6517    343115 deny tcp from any to any established
00600      0         0 deny log logamount 50 ip from any to any ipoptions rr
00610      0         0 deny log logamount 50 ip from any to any ipoptions ts
00620      0         0 deny log logamount 50 ip from any to any ipoptions lsrr
00630      0         0 deny log logamount 50 ip from any to any ipoptions ssrr
00640      0         0 deny log logamount 50 tcp from any to any tcpflags syn,fin
00650      0         0 deny log logamount 50 tcp from any to any tcpflags syn,rst
00700  21328   6139067 allow ip from any to any out keep-state
00800    154     10734 allow icmp from any to any
00900      0         0 deny ip from table(10) to any
10001      0         0 allow tcp from table(2) to any dst-port 21 setup limit src-addr 30
10002      0         0 allow tcp from table(2) to any dst-port 22 setup limit src-addr 24
10003      0         0 allow tcp from table(2) to any dst-port 25 setup limit src-addr 30
10004      0         0 allow tcp from table(2) to any dst-port 587 setup limit src-addr 60
10005      0         0 allow tcp from table(2) to any dst-port 53 setup limit src-addr 9
10006      0         0 allow udp from table(2) to any dst-port 53 limit src-addr 9
10007      0         0 allow tcp from table(2) to any dst-port 80 setup limit src-addr 80
10008      0         0 allow tcp from table(2) to any dst-port 110 setup limit src-addr 60
10009      0         0 allow tcp from table(2) to any dst-port 143 setup limit src-addr 30
10010      0         0 allow tcp from table(2) to any dst-port 443 setup limit src-addr 30
10011      0         0 allow tcp from table(2) to any dst-port 2222 setup limit src-addr 36
10012      0         0 allow tcp from table(2) to any dst-port 35000-35999 in setup limit src-addr 30
10013      0         0 allow tcp from table(2) to any dst-port 993 setup limit src-addr 30
10014      0         0 allow tcp from table(2) to any dst-port 995 setup limit src-addr 30
10015      0         0 allow tcp from table(2) to any dst-port 465 setup limit src-addr 30
10016      0         0 allow tcp from table(2) to any dst-port 585 setup limit src-addr 30
60001  16410   1123899 allow tcp from any to any dst-port 21 setup limit src-addr 20
60002  19708   2537252 allow tcp from any to any dst-port 22 setup limit src-addr 16
60003  17376  11550490 allow tcp from any to any dst-port 25 setup limit src-addr 10
60004      0         0 allow tcp from any to any dst-port 587 setup limit src-addr 20
60005      0         0 allow tcp from any to any dst-port 53 setup limit src-addr 3
60006      0         0 allow udp from any to any dst-port 53 limit src-addr 3
60007 278291 177549716 allow tcp from any to any dst-port 80 setup limit src-addr 40
60008  14174   5820292 allow tcp from any to any dst-port 110 setup limit src-addr 20
60009   1662    391070 allow tcp from any to any dst-port 143 setup limit src-addr 10
60010     18       752 allow tcp from any to any dst-port 443 setup limit src-addr 10
60011      0         0 allow tcp from any to any dst-port 2222 setup limit src-addr 12
60012  35926  26126542 allow tcp from any to any dst-port 35000-35999 in setup limit src-addr 10
60013  13809   5885741 allow tcp from any to any dst-port 993 setup limit src-addr 10
60014      0         0 allow tcp from any to any dst-port 995 setup limit src-addr 10
60015      0         0 allow tcp from any to any dst-port 465 setup limit src-addr 10
60016      0         0 allow tcp from any to any dst-port 585 setup limit src-addr 10
60116  38702   6282861 deny log ip from any to any
65535      0         0 deny ip from any to any
 
Top