Blocking bots in nginx_apache mode

[sunsunshining]

I tried to block some bots in a mixed nginx_apache mode by adding the following block to "nginx.conf proxy" for a domain via custom-httpd → Customize → Paste below code in first box → Save → DA Build rewrite configs.

But this did not seem to block them when checking the log.

if ($http_user_agent ~* (Amazonbot|anthropic-ai|Applebot|Applebot-Extended|AwarioRssBot|AwarioSmartBot|Bytespider|CCBot|ChatGPT|ChatGPT-User|Claude-Web|ClaudeBot|cohere-ai|DataForSeoBot|Diffbot|FacebookBot|Google-Extended|GPTBot|ImagesiftBot|magpie-crawler|omgili|Omgilibot|peer39_crawler|PerplexityBot|SemrushBot|Yandex|YandexBot|YouBot)) {
        return 403;
    }

 

[exlhost]

You need to add it to apache as well

 

[sunsunshining]

You mean use the same code in the Apache config file for the specific site? I already have it as mod_rewrite directive in .htaccess but that does not appear to be applied.

 

[Ohm J]

maybe you put in wrong input box, which place you put ? Custom, Custom1, Custom2, Custom3, Custom4 ?

 

[sunsunshining]

in the top box for both nginx and apache, is this wrong?

 

[Ohm J]

try put in Custom2. you can see the full template and find "|CUSTOM|" text, it's will show the section will be customize.

just only put in nginx, it's fine.

 

[sunsunshining]

That is what I did, I could not see any |CUSTOM| text. Here is a snippet (one of 14 instances of if ($http_user_agent...)

server
{
    listen ;
    listen [2001:41d0:700:2bd5:0:0:0:0]:80;
    server_name domain.com www.domain.com;
    access_log /var/log/nginx/domains/domain.com.log;
    access_log /var/log/nginx/domains/domain.com.bytes bytes;
    error_log /var/log/nginx/domains/domain.com.error.log;
    root "/home/translat/domains/domain.com/public_html";
    index index.php index.html index.htm;
    # Mail auto configuration (Thunderbird)
    location = "/.well-known/autoconfig/mail/config-v1.1.xml" {
        proxy_pass http://unix:/usr/local/directadmin/shared/internal.sock;
        proxy_set_header X-Forwarded-For  $remote_addr;
        proxy_set_header X-Forwarded-Host $host;
    }
    location /
    {
if ($http_user_agent ~* (Amazonbot|anthropic-ai|Applebot|Applebot-Extended|AwarioRssBot|AwarioSmartBot|AwarioBot|Bytespider|CCBot|ChatGPT|ChatGPT-User|Claude-Web|ClaudeBot|cohere-ai|DataForSeoBot|Diffbot|FacebookBot|Google-Extended|GPTBot|ImagesiftBot|magpie-crawler|omgili|Omgilibot|peer39_crawler|PerplexityBot|YouBot|Yandex|YandexBot)) {
        return 403;
    }
        # access_log off;
        proxy_buffering off;
        proxy_pass http://;
        proxy_set_header X-Client-IP      $remote_addr;
        proxy_set_header X-Accel-Internal /nginx_static_files;
        proxy_set_header Host             $host;
        proxy_set_header X-Forwarded-For  $proxy_add_x_forwarded_for;
        proxy_hide_header Upgrade;
    }

 

[Ohm J]

that's should work, maybe trying reload the nginx service.

I'm talking about template, not any users virtualhost config. it's should be in the same webpage.

p.s. if you accident posting your server IPs, please remove it.