Apache 2.2 ddos multiple IPs

[Rich-Boy]

Hi all, so this is the usual scenario some of you are probably used to seeing... someone is flooding port 80 on my server although the botnet is quite big and every IP connects only once. In my server-status page i see lots of this...

Total accesses: 6356 - Total Traffic: 31.2 MB
CPU Usage: u55.3984 s9.75781 cu.0078125 cs0 - 6.03% CPU load
5.88 requests/sec - 29.6 kB/second - 5.0 kB/request
150 requests currently being processed, 0 idle workers

RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRCRRRRRRRRRRCRRRRRRRRRRRRRRRRRRRRRR
RRRRRRRRRRRRRRRRWRRRRRRRWRRCCRRRRRRRRRRRRRRRRRRRRRRRRRRRRWRRWRRR
RRRRRRRWRRRRWRRRRRRRRR..........................................
................................................................

Scoreboard Key:
"_" Waiting for Connection, "S" Starting up, "R" Reading Request,
"W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
"C" Closing connection, "L" Logging, "G" Gracefully finishing,
"I" Idle cleanup of worker, "." Open slot with no current process

Srv	PID	Acc	M	CPU 	SS	Req	Conn	Child	Slot	Client	VHost	Request
0-1	30786	0/19/19	R 	1.52	14	0	0.0	0.17	0.17 	?	?	..reading..
1-1	30787	0/37/37	R 	0.03	58	0	0.0	0.18	0.18 	?	?	..reading..
2-1	30788	0/22/22	R 	0.18	2	0	0.0	0.04	0.04 	?	?	..reading..
3-1	30789	0/31/31	R 	0.63	53	0	0.0	0.18	0.18 	?	?	..reading..
4-1	30790	0/57/57	R 	0.69	28	0	0.0	0.19	0.19 	?	?	..reading..
5-1	30791	0/25/25	R 	0.38	28	0	0.0	0.07	0.07 	?	?	..reading..
6-1	30792	0/17/17	R 	0.40	59	0	0.0	0.01	0.01 	?	?	..reading..
7-1	30793	0/12/12	R 	0.20	11	0	0.0	0.00	0.00 	?	?	..reading..
8-1	30794	0/103/103	R 	0.16	27	0	0.0	0.72	0.72 	?	?	..reading..
9-1	30795	0/15/15	R 	0.23	0	86	0.0	0.14	0.14 	?	?	..reading..
10-1	30796	0/21/21	R 	0.34	60	0	0.0	0.03	0.03 	?	?	..reading..
11-1	30797	0/19/19	R 	0.29	29	0	0.0	0.01	0.01 	?	?	..reading..
12-1	30798	0/65/65	R 	0.19	60	0	0.0	0.07	0.07 	?	?	..reading..
13-1	30799	0/9/9	R 	0.18	12	0	0.0	0.05	0.05 	?	?	..reading..
14-1	30800	0/47/47	R 	0.13	45	0	0.0	0.44	0.44 	?	?	..reading..
15-1	30801	0/87/87	R 	0.19	35	1	0.0	0.35	0.35 	?	?	..reading..
16-1	30802	0/36/36	R 	0.70	0	0	0.0	0.06	0.06 	?	?	..reading..
17-1	30803	0/26/26	R 	0.56	58	113	0.0	0.04	0.04 	?	?	..reading..
18-1	30804	0/69/69	R 	0.34	28	0	0.0	0.39	0.39 	?	?	..reading..
19-1	30805	0/11/11	R 	0.24	10	0	0.0	0.10	0.10 	?	?	..reading..
20-1	30806	0/52/52	R 	0.55	59	1	0.0	0.12	0.12 	?	?	..reading..
21-1	30807	0/20/20	R 	0.56	60	141	0.0	0.05	0.05 	?	?	..reading..
22-1	30808	0/107/107	R 	0.45	10	25	0.0	0.85	0.85 	?	?	..reading..
23-1	30809	0/15/15	R 	0.59	60	0	0.0	0.13	0.13 	?	?	..reading..
24-1	30810	0/21/21	R 	0.46	60	0	0.0	0.06	0.06 	?	?	..reading..
25-1	30811	0/78/78	R 	0.52	37	0	0.0	0.07	0.07 	?	?	..reading..
26-1	30812	0/42/42	R 	0.16	35	1	0.0	0.05	0.05 	?	?	..reading..
27-1	30813	0/20/20	R 	0.47	51	0	0.0	0.05	0.05 	?	?	..reading..
28-1	30814	0/42/42	R 	0.27	36	0	0.0	0.26	0.26 	?	?	..reading..
29-1	30815	0/13/13	R 	0.34	10	0	0.0	0.06	0.06 	?	?	..reading..

My server is quite popular anyway so i'm desperate to figure out which IPs are legit and which are flooding me, im using FreeBSD 6.2 and i've tried syn rating which didnt help much at all. The only option i do have is increasing maxclients so that apache doesnt get overloaded but this seems to just avoid the problem rather than tackle it.
Basically i need a way of identifying bad ips from good so my software firewall can deal with them, does anyone have any ideas?

 

[Randy]

Possibly could Modevasive help you? (See update.script)

 

[Rich-Boy]

Ah yea i forgot to mention that i already have that installed but it only bans IPs about once every half an hour... it doesn't have much of an effect maybe partially because it wasn't designed for apache 2.2?

Heres my current mod_evasive settings anyway;

<IfModule mod_evasive20.c>
DOSHashTableSize 3097
DOSPageCount 2
DOSSiteCount 50
DOSPageInterval 1
DOSSiteInterval 1
DOSBlockingPeriod 10
</IfModule>

Could i make it more agressive so maybe it would be of better use, if so how?

 

[andyreed]

Hi all, so this is the usual scenario some of you are probably used to seeing... someone is flooding port 80 on my server although the botnet is quite big and every IP connects only once. In my server-status page i see lots of this...

It sounds that you're under SYn Flooding attack. This article explains different Internet attacks: http://servertune.com/kbase/entry/64/

The only option i do have is increasing maxclients so that apache doesnt get overloaded but this seems to just avoid the problem rather than tackle it.

That won't help because SYN flooding attack consumes your system resources until no more incoming TCP :80 connections are possible.

Basically i need a way of identifying bad ips from good so my software firewall can deal with them, does anyone have any ideas?

The only possible way, if you are positive that you're under TCP SYN Flooding attack is to get top noch hardware-base firewall like Cisco ATA 5020. Mod Easive, Mod Security or any software-based firewall will help in this case. A combination of a software-base and a hardware-based firewalls is the ideal solution in this case.

Run this command and see if you're under Syn attack:

/bin/netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr

 

[Rich-Boy]

The thing is when i do increase maxclients from the default 150 to something higher like 300 the system returns to it's lightning fast response times even though its still being attacked. When the 'requests currently being processed' value stays below the maxclients value (i set this to 300 and everything was ok) the server responds fine, this suggests to me that if more bots got involved with the attack my server would get bogged down again until i increased maxclients further.

Here's the output from that netstat syntax;

  93 0
   1 cc155000
   1 cb13e550
   1 c69cadd0
   1 c69c9aa0
   1 c69c6990
   1 c6966880
   1 c6966770
   1 c690b000
   1 Inode
   1

Maybe it doesn't make as much sense as it does in linux as im using FreeBSD?