Using CSF with IPSET, what is a good configuration?

Richard G

Richard G

Verified User
Joined
Jul 6, 2008
Messages
13,391
Location
Maastricht
I'm just wondering. We want to increase the option to have more ip's blocked.

But except for enabling ipset, what are decent settings to use for DENY_IP_LIMIT and DENY_TEMP_IP_LIMIT for example?
Because I know if you go higher than 1000 in DENY_TEMP_IP_LIMIT then when restaring CSF it gives a warning that recommended is between 100 and 1000.
Does the TEMP limit also make use of IPSET?

Also I've seen that LF_IPSET_MAXELEM = "65536" which some site advised to increase to 16777216, seems a bit much to me, what value do you use?

It it still possible to use the normal csf commands like csf -d or csf -rd etc. to add and remove ip's?

Any other good advise on using ipset?
 
Just answered by PM. But as you told you are on Apache, well, my values are good under Litespeed, I doubt it would work so fast under apache :)
 
I've set it to 16777216 since 4weeks, no problems so far.

In our systems we've blocked about 200k IPs, since we use commercial blocklists (only since 4 days in use) the serverload has decreased up to 80%, especially because all the WP login, rpc & comment attacks are topped. Electricity costs have also decreased.

The removal of IPs (with csf -rd) which were blocked by the system itself does not always work, why is unclear to me, csf -ra fixes it then.

RAM consumption on the other hand went up by 4GB on all systems, but that doesn't matter. In proxmox containers ipset never really worked.

For me personally, the conclusion is that i should have done this much earlier.
 
johannes said:
Just answered by PM.
Click to expand...
Yep, thank you Johannes.

apogee said:
RAM consumption on the other hand went up by 4GB on all systems
Click to expand...
Oh lol, only 4 GB with 200K ip's, that's nothing. We have 64GB Ram and still 42 GB free and 59 available. So loosing 4 is no issue there.
Thank you.

What exactly does that setting 16777216 do? Because I don't really understand what it does.
 
16777216 is the limit of IP entries in ipset, if there are more then these are simply not read (error: "Hash is full, cannot add more elements"). How the 4GB ram consumption exactly come about I have not researched further, but for us no problem, each box has at least 265GB in it and no uses the whole memory.

Presumably you mean it sacastically but for me it does not matter if I need 4GB more because I have them anyway and the load and power consumption decrease and not to forget we save time because the servers do not have to make pointless RBL requests, we have also loaded payd RBL lists into ipset, so the logs also are no longer filled so much, saves IO on the storages.

Example of a customer wordpress site, plugin "Limit Login Attempts Reloaded":

da-wp-7.png


Another example after we removed a list and bought a commercial ones:


da-wp-8.png

Yesterday evening somehow 50k IPs disappeared from the lists, I'm still clarifying why, but you can see immediately how the login attempts rose again.


Here the apache load with and without blocklists (other server), was installed one day later:

apache_volume-week.png

Currently I'm playing around with it every day to optimize it, oh well meanwhile the setup is on all productive servers, so far not a single complaint from a customer....
 
Would you mind to share which commercial list you are using? Or maybe by PM? I did a lot of comparisons but never got to a point. Thank you.
 
apogee said:
Presumably you mean it sacastically but for me it does not matter if I need 4GB more because
Click to expand...
Why do you think I might be sarcastically? Sorry if I gave that impression to you somehow.

As stated I was amused that with so many ip's for ipset only 4 GB ram was needed, which ment it would not get us into trouble with our resources.
So I was amused that I didn't needed to be worried and things needed way less then I expected.

Thanks for the graphs with the information, really interesting, and also for people reading this thread i guess.

apogee said:
16777216 is the limit of IP entries in ipset
Click to expand...
Oke so in fact this is only needed if we would run into the error you mentioned with our 65535 setting. Thank you.
 
Richard G said:
I'm just wondering. We want to increase the option to have more ip's blocked.

But except for enabling ipset, what are decent settings to use for DENY_IP_LIMIT and DENY_TEMP_IP_LIMIT for example?
Because I know if you go higher than 1000 in DENY_TEMP_IP_LIMIT then when restaring CSF it gives a warning that recommended is between 100 and 1000.
Does the TEMP limit also make use of IPSET?

Also I've seen that LF_IPSET_MAXELEM = "65536" which some site advised to increase to 16777216, seems a bit much to me, what value do you use?

It it still possible to use the normal csf commands like csf -d or csf -rd etc. to add and remove ip's?

Any other good advise on using ipset?
Click to expand...
If you've enabled IPSET support in CSF (and kernel supports it), the csf commands should work fine.

You can check each IPSET chain set's memory usage and number of entries using
Bash: 
ipset list -t
 
Oh nice, thanks!

eva2000 said:
memory usage and number of entries using
Click to expand...
That command is also really usefull.

So when I have this output:
Code: 
Name: chain_DENY
Type: hash:net
Revision: 6
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 30648
is that read in bytes or in kbytes? So is this 30 kb or 30 mb? I presume 30 kb because I've got another one over 50 (tor stuff). :)

Oh yes I don't understand this. All 3 servers have same csf configuration, now I see this:
Code: 
Name: bl_CIARMY
Type: hash:net
Revision: 6
Header: family inet hashsize 4096 maxelem 65536
Size in memory: 340472
References: 1
Number of entries: 14991

But on the other server it's more like this:
Code: 
Name: bl_CIARMY
Type: hash:net
Revision: 6
Header: family inet hashsize 8192 maxelem 65536
Size in memory: 463032
References: 1
Number of entries: 14991

The 3rd server is the same, only different size in memory and he has 14999 entry's. :)

Odd thing here is that 1 server has 4096 hashsize and the others 8192, while it's exactly the same blocklist.
Where could this difference in hashsize come from?
 
Last edited:
AFAIK memory is reported in bytes.

As to hashsize not sure, different servers of mine report differently even if CSF Firewall's LF_IPSET_HASHSIZE is set to 1024

Bash: 
grep -B4 -A6 HASHSIZE /etc/csf/csf.conf
# The following sets the hashsize for ipset sets, which must be a power of 2.
#
# Note: Increasing this value will consume more memory for all sets
# Default: "1024"
LF_IPSET_HASHSIZE = "1024"

# The following sets the maxelem for ipset sets.
#
# Note: Increasing this value will consume more memory for all sets
# Default: "65536"
LF_IPSET_MAXELEM = "524288"

Bash: 
ipset list -t
Name: chain_DENY
Type: hash:net
Revision: 6
Header: family inet hashsize 32768 maxelem 524288
Size in memory: 1823480
References: 2
Number of entries: 58777

edit: this suggests hashsize can grow dynamically to accommodate larger sets https://github.com/mighq/puppet-ipset/issues/14
 
Last edited:
johannes said:
Would you mind to share which commercial list you are using? Or maybe by PM? I did a lot of comparisons but never got to a point. Thank you.
Click to expand...
U got a PM.


I'm totally busy at the moment, I'll come back to the topic on thursday!
 
eva2000 said:
As to hashsize not sure, different servers of mine report differently even if CSF Firewall's LF_IPSET_HASHSIZE is set to 1024
Click to expand...
Ah oke thank you, mine are also set to 1024 on every server. I should have gone over to Ipset a while ago I see.
I'm not native English so it seems lange hashsizes produce errors. So it's not quite clear to my if we should raise the size or not. I presume not as it sizes dynamically, so it won't throw errors?

apogee said:
sounds sarcastic to me but English is not my first language, so I may have misunderstood.
Click to expand...
English is not my first language either. It just means that 4 GB use is really not a lot of memory for so many ip blocks. Unless i'm mistaken and less memory should be used in fact.
I don't know if you're also Dutch. In that case... Net als met prijzen, 1,50 is niks voor een kop koffie tegenwoordig, 4 Gb geheugen is niks voor 200.000 ip blocks.
Anyway, sorry again, it is in no way ment to be or to sound sarcastic. I was just being happy that many blocks did not use a lot of memory.
 
Really stupid, I took that ironically ;-)

I can not interpret everything correctly, my English teacher in vocational school was a sympathetic Hungarian, which also did not master it perfectly.

Unfortunately I am not Dutch, but I know many, very dear people, I was also often in Holland, especially at the North Sea, I understand it and can read it but not speak, I speak 5 (badyl - as my english) languages but not yours.

I have attacked you a few times, unconsciously, I'm sorry, please forgive me. It was obviously only due to the understanding
 
apogee said:
Really stupid, I took that ironically ;-)
Click to expand...
Nah that's not stupid. Language things can happen, better to say it so one can explain, then maybe have unneeded irritations. So I'm glad you said it so I could explain.

apogee said:
I have attacked you a few times, unconsciously, I'm sorry, please forgive me.
Click to expand...
You did? I'm not aware of that.
Only thing I've seen is you stating you had a certain probable impression, which prooved wrong after I could explain. Nothing wrong with that, I never felt attacked, so no worries. But thank you for the nice reply!
 
Hi, I was reading and and I'm curious how do you increase the number of entries in ipset? it seems like I'm only limited to 200 entries.

Code: 
Revision: 6
Header: family inet hashsize 1024 maxelem 655360
Size in memory: 9384
References: 2
Number of entries: 200

Name: chain_6_DENY
Type: hash:net
Revision: 6
Header: family inet6 hashsize 1024 maxelem 655360
Size in memory: 1240
References: 2
Number of entries: 0

Name: chain_ALLOW
Type: hash:net
Revision: 6
Header: family inet hashsize 1024 maxelem 655360
Size in memory: 888
References: 2
Number of entries: 9

Name: chain_6_ALLOW
Type: hash:net
Revision: 6
Header: family inet6 hashsize 1024 maxelem 655360
Size in memory: 1240
References: 2
Number of entries: 0
 
The number of blocks you mean? I've done that by changing this setting to a higher value, it's 5000 at the moment.
DENY_IP_LIMIT = "5000"
 
Ahh sorry I am not familiar with all these stuff. says here if we wish to set to a higher value, they recommend using IPSET. My thinking was that by activating IPSET and ipset_maxelem will increase the block count. So if I increase the deny limit, it will still use ipset list?

# For implementations wishing to set this value significantly higher, we
# recommend using the IPSET option
DENY_IP_LIMIT = "200"
 
Well that did the trick thank you. I did see this option when I was looking but I was afraid that the warning meant to use IPSET instead of increasing the value; and by using IPSET it will use a separate list from ipset with its own limits.
 
You must log in or register to reply here.
Back
Top