Amazon bots invasion. How to stop?

dorucrisan

dorucrisan

Verified User
Joined
Oct 23, 2021
Messages
225
Location
Bucharest / Romania
I have an invasion of amazon bots on my server, hundreds. I checked the IP's, all are amazon bots located in various states of the USA. I can't ban USA as country, as I need googlebots and maybe other services. Those bots seems to like one particular web shop located on this server. Any ideea how to stop them? I made a list manually and added to csf/csf.deny but the list is endless. I ban 20, other 50 take their place. They are searching the website using filters and kills MySQL. CPU is 400-500% in "top". Any solutions? Thanks.
 
Have look in this thread, starting post 5 and further.
V

Thread 'One of the websites gets extreme bandwidth'

Hey guys,
I have a website that gets 100Gb of traffic but this isn't a busy website. Its a WordPress and it's completely clean and protected with totally blocking the login page (except for my IP). I have checked every single file on that website manually (images too), in hope I can find anything suspicious but nothing.
There is a firewall installed and its capturing a few blocks a day. Still the website gets a lot of bandwidth.
Is there a way in Direct Admin to see an exact file activity for the website?
 
You could (temporarely) block Amazon`s ASN AS16509 https://ipinfo.io/AS16509 in the CSF firewall (there where you put country codes in). But check the load and the sitespeed afterwards, dont know if such a big list could slow your server down.
 
Thanks for the information. Is anybody willing to help me for a fee to solve this problem? Here is a short list of what I found, there are hundreds. I don't even know if they are really amazonbots or some other bots on their AWS.

52.167.144.236
3.141.18.167
18.222.23.166
18.116.37.17
18.216.194.71
3.21.168.188
18.222.170.43
3.137.169.218
3.135.240.57
3.148.109.137
All looks like this:

1744617533216.png
 
There are situations where we temporarily block all AWS traffic. Here's how we do it:

Code: 
# wget https://ip-ranges.amazonaws.com/ip-ranges.json

Code: 
# grep ip_prefix ip-ranges.json | awk '{print $2}' | tr -d \", >>/etc/csf/csf.deny

Code: 
# csf -r
 
Last edited:
Remco00 said:
There are situations where we temporarily block all AWS traffic. Here's how we do it:

Code: 
# wget https://ip-ranges.amazonaws.com/ip-ranges.json

Code: 
# ip-ranges.json | awk '{print $2}' | tr -d \", >>/etc/csf/csf.deny

Code: 
# csf -r
Click to expand...
Thank you. Where do I code that? Login to server with SSH and that's it?
Thanks so much Remco00. Is that OK? It says "command not found".

1744624867199.png
 
Last edited:
Sorry, my mistake. Use this:

Code: 
# grep ip_prefix ip-ranges.json | awk '{print $2}' | tr -d \", >>/etc/csf/csf.deny
 
Try changing the CT_LIMIT value in /etc/csf/csf.conf value to 150 and restart CSF.
 
you guy forgot to restart firewall after adding the blocklist.

Code: 
csf -ra

note# if website behind cloudflare or other CDN service, server firewall not working at all.
 
Remco00 said:
You’re welcome. Glad I could help you.
Click to expand...
Heeeeelp!
Sorry, something weird happened. I coded as instructed, all good. Over 8000 IP's were added to csf.deny and all was good. However this morning, I found the CPU again in overload, checked that csf.deny file and now it only has 170 lines or so. I checked 2 other servers, same happened. Any ideea what could be the cause? Here it is, new and old csf.deny. Owner is root.
1744790200439.png
1744790255844.png

1744791125350.png
 
Last edited:
Check the value of DENY_IP_LIMIT and (depending on the resources of your server) increase it.
 
dorucrisan said:
I have an invasion of amazon bots on my server, hundreds. I checked the IP's, all are amazon bots located in various states of the USA. I can't ban USA as country, as I need googlebots and maybe other services. Those bots seems to like one particular web shop located on this server. While reviewing traffic to pages about wholesale suppliers for amazon sellers, I noticed the bot activity increasing significantly. Any ideea how to stop them? I made a list manually and added to csf/csf.deny but the list is endless. I ban 20, other 50 take their place. They are searching the website using filters and kills MySQL. CPU is 400-500% in "top". Any solutions? Thanks.
Click to expand...
They're most likely not Amazon bots but bots running on AWS. Instead of chasing IP addresses, enable rate limiting, use Fail2Ban or Cloudflare to block abusive requests, and optimize or cache the search/filter pages that are overloading MySQL. Blocking by behavior is far more effective than maintaining an endless IP blacklist.
 
kingstondouglas said:
They're most likely not Amazon bots but bots running on AWS. Instead of chasing IP addresses, enable rate limiting, use Fail2Ban or Cloudflare to block abusive requests, and optimize or cache the search/filter pages that are overloading MySQL. Blocking by behavior is far more effective than maintaining an endless IP blacklist.
Click to expand...
Yes that is correct thank you, solved with Cloudflare and some rules. All OK now.
 
You must log in or register to reply here.
Back
Top