Awstat to read nginx logs when using nginx+apache

zEitEr

Super Moderator
Joined
Apr 11, 2005
Messages
15,366
Location
www.poralix.com
Hello,

We've got an issue with bandwidth usage shown in Awstats.

Images (static files) requested through nginx/apache are fully logged with its size only in nginx logs:


nginx shows size 2084443 bytes :
Code:
207.46.13.36 - - [06/Nov/2015:09:52:25 +0100] "GET /Image/1602/16.JPG HTTP/1.1" 200 2084443 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"


apache shows size only 310 bytes (tech headers for nginx):
Code:
207.46.13.36 - - [06/Nov/2015:09:52:19 +0100] "GET /Image/1602/16.JPG HTTP/1.0" 200 310 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"


it's due to how nginx is proxying requests for static files:

Code:
        location /
        {
                # access_log off;
                proxy_buffering off;
                proxy_pass http://11.22.33.44:8080;
                proxy_set_header X-Client-IP      $remote_addr;
                proxy_set_header X-Accel-Internal /nginx_static_files;
                proxy_set_header Host             $host;
                proxy_set_header X-Forwarded-For  $proxy_add_x_forwarded_for;
        }
        location /nginx_static_files/
        {
                # access_log  /var/log/nginx/access_log_proxy;
                alias       /home/userbob/domains/domain.com/public_html/;
                internal;
        }

Thus we have the following stats:


Date 1:
From apache logs Total = 665.337 MB / 588798 hits
From nginx logs Total = 17786.5 MB / 588689 hits


Images from (/Image/) requested 242632 times according to apache logs: Total = 74.4036 MB
Images from (/Image/) requested 242889 times according to NGINX logs: Total = 16735.7 MB




Date 2:
From apache logs Total = 360.118 MB / 263632 hits
From nginx logs Total = 12664.1 MB / 264290 hits


Images from (/Image/) requested 175191 times according to apache logs: Total = 53.7159 MB
Images from (/Image/) requested 175807 times according to NGINX logs: Total = 11986.4 MB




Awstats gets data from apache logs only, and it does not read nginx logs. And it is nginx which serves static files. Thus Awstat might need to get data from nginx logs.

The same might be actual for webalizer.

Please fix it.
 
As a quick solution run this:

Code:
perl -pi -e 's|/var/log/httpd/domains/|/var/log/nginx/domains/|' /home/*/domains/*/awstats/.data/awstat*.conf

to make awstats to read nginx logs instead of apache.

Run this:

Code:
perl -pi -e 's|/var/log/nginx/domains/|/var/log/httpd/domains/|' /home/*/domains/*/awstats/.data/awstat*.conf

to roll back and make awstats to read apache logs instead of nginx.
 
Thanks for the report.
I've checked awstats_process.sh and it shows this:
Code:
HTTPD=httpd
if [ "`/usr/local/directadmin/directadmin c | grep ^nginx= | cut -d= -f2`" -eq 1 ]; then
        HTTPD=nginx
fi
which is likely what we need to change.

I've added this code chunk just below it, changing it to this:
Code:
HTTPD=httpd
if [ "`/usr/local/directadmin/directadmin c | grep ^nginx= | cut -d= -f2`" -eq 1 ]; then
    HTTPD=nginx
fi
if [ "`/usr/local/directadmin/directadmin c | grep ^nginx_proxy= | cut -d= -f2`" -eq 1 ]; then
    HTTPD=nginx
fi
so give that a try and let me know if it doesn't work.

Thanks!
John
 
Thank you for a quick reply, John.

The code seems to be working fine and new config is created with LogFile= pointed to nginx logfile.

So suggested by you modification will only create awstats.domain.com.conf with LogFile= option pointed to nginx's log. And we still need to change existing configs like it's mentioned in the post#2. Correct? Or will you add the lines into CustomBuild script and the changes will be applied alongside with switching to nginx+apache?
 
It wouldn't change existing configs, so would still have the old paths.

However, I don't thing that entries really matters as awstats cannot access the apache logs anyway.
The data is only updated with the nightly tally, and the cgi version just chews on that comupted data, not the live logs.

John
 
Back
Top