TwitterBot visits

Peter Laws

Verified User
Joined
Sep 13, 2008
Messages
1,747
Location
London UK
Anyone know if we can limit TwitterBot visits for "card" refreshing? It is hammering one of my client's sites (~144 connections), bandwidth going through the roof requesting about ~25 images every ~3 seconds, not to mention it's also making php-fpm choke...

Luckily it only happens every few days.
 

wattie

Verified User
Joined
May 31, 2008
Messages
1,001
Location
Bulgaria
If you like to stop it and leave all other bots active, you can add this in robots.txt:

User-agent: Twitterbot
Disallow: /

User-agent: *
Disallow:
I am unsure if your customer will be happy with such solution - he may want to be indexed :)
 
Top