Results 1 to 2 of 2

Thread: TwitterBot visits

  1. #1
    Join Date
    Sep 2008
    Location
    London UK
    Posts
    1,654

    Question TwitterBot visits

    Anyone know if we can limit TwitterBot visits for "card" refreshing? It is hammering one of my client's sites (~144 connections), bandwidth going through the roof requesting about ~25 images every ~3 seconds, not to mention it's also making php-fpm choke...

    Luckily it only happens every few days.

  2. #2
    Join Date
    May 2008
    Posts
    755
    If you like to stop it and leave all other bots active, you can add this in robots.txt:

    User-agent: Twitterbot
    Disallow: /

    User-agent: *
    Disallow:
    I am unsure if your customer will be happy with such solution - he may want to be indexed

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •