TwitterBot visits

LawsHosting

Verified User
Joined
Sep 13, 2008
Messages
2,365
Location
London UK
Anyone know if we can limit TwitterBot visits for "card" refreshing? It is hammering one of my client's sites (~144 connections), bandwidth going through the roof requesting about ~25 images every ~3 seconds, not to mention it's also making php-fpm choke...

Luckily it only happens every few days.
 
If you like to stop it and leave all other bots active, you can add this in robots.txt:

User-agent: Twitterbot
Disallow: /

User-agent: *
Disallow:

I am unsure if your customer will be happy with such solution - he may want to be indexed :)
 
Back
Top