Help with blocking bad bots but allowing Googlebot?

M

maxb1999

Verified User
Joined
Dec 27, 2024
Messages
12
Hi all,

I'm running a server with several WordPress websites, and lately it's been getting hit by a huge number of bots. The traffic is so intense that it's causing the entire server to slow down or even go down completely.

I’ve tried blocking all bots, but when I do that, the Google Ads bot also gets blocked—which is obviously a problem.

I’ve read the documentation from Google about verifying Googlebot, but I'm struggling to find a practical and efficient way to implement this at the server level, especially across multiple sites.

Has anyone found a good way to:
  • Block most bad bots,
  • But still allow only the real Googlebot (and maybe other important crawlers like Bing or AdsBot)?

Any tips, tools, firewall rules, or configs you use would be much appreciated!

Thanks in advance!
 
Hello.

Please have a look here, various bot blocking options are already mentioned there.
V

Thread 'One of the websites gets extreme bandwidth'

Hey guys,
I have a website that gets 100Gb of traffic but this isn't a busy website. Its a WordPress and it's completely clean and protected with totally blocking the login page (except for my IP). I have checked every single file on that website manually (images too), in hope I can find anything suspicious but nothing.
There is a firewall installed and its capturing a few blocks a day. Still the website gets a lot of bandwidth.
Is there a way in Direct Admin to see an exact file activity for the website?
 
You must log in or register to reply here.
Back
Top