TwitterBot visits

Peter Laws

Verified User
Anyone know if we can limit TwitterBot visits for "card" refreshing? It is hammering one of my client's sites (~144 connections), bandwidth going through the roof requesting about ~25 images every ~3 seconds, not to mention it's also making php-fpm choke...

Luckily it only happens every few days.
 

wattie

Verified User
If you like to stop it and leave all other bots active, you can add this in robots.txt:

User-agent: Twitterbot
Disallow: /

User-agent: *
Disallow:
I am unsure if your customer will be happy with such solution - he may want to be indexed :)
 
Top