Apache/Nginx bytes logs

[hansmiddelhoek]

For every domain there is a /var/log/(httpd|nginx)/domains/domain.tld.bytes log which is rotated every day. How can I configure the amount of bytes logs that I want to keep? It seems related to logs_to_keep and I think that the bytes logs are kept twice the value set at logs_to_keep. Example: I set logs_to_keep to 7 on a server and I have 14 domain bytes logs. Another server where I set logs_to_keep to 21 I have 42 domain bytes logs. This takes unnecessary disk space and I would like to be able to set the rotation amount for these bytes logs. Is it necessary to keep more than 1? Maybe a new directadmin.conf value byteslogs_to_keep? Where are the historic bytes logs used for, eg. why do we need /var/log/httpd/domains/domain.tld.bytes.9?

 

[MaXi32]

I would also like to know how to adjust the log rotation settings for .bytes files in DirectAdmin. I don't think the logs_to_keep option is used for rotating .bytes files. By default, it seems that DirectAdmin stores up to 10 rotated .bytes files for each domain, with the last index being .bytes.9. It would be helpful if perhaps an option called byteslogs_to_keep can manage this like you just mentioned.

Is it necessary to keep more than 1?

Just like other .log files get rotated it is about saving disk space, and for .bytes file if you are auditing server performance keep it (I see no other purpose to keep more than 1). However, there should be an option to disable .bytes logging altogether (some people don't need it), but I cannot find this option in DirectAdmin or understand how Directadmin manages the .bytes file at this time.

 

[hansmiddelhoek]

I don't think the logs_to_keep option is used for rotating .bytes files. By default, it seems that DirectAdmin stores up to 10 rotated .bytes files for each domain, with the last index being .bytes.9

Maybe not. But the default for logs_to_keep is 5 and 10 .bytes files would be twice that amount. Check your config with:

da config-get logs_to_keep

AFAIK the .bytes files are only used for calculating the bandwidth used by the domain. The DA nightly tally processes these files and rotates them.

The .bytes files are created by Nginx and Apache:

/etc/nginx/nginx-defaults.conf:log_format bytes '$bytes_sent $request_length';
/etc/httpd/conf/httpd.conf:      LogFormat "%O %I" bytes

And in /usr/local/directadmin/data/users/<USER>/nginx.conf

access_log /var/log/nginx/domains/<DOMAIN>.bytes bytes;

And in /usr/local/directadmin/data/users/<USER>/httpd.conf

CustomLog /var/log/httpd/domains/<DOMAIN>.bytes bytes

You can make a custom template of the Nginx/Apache configuration files and remove the bytes log line but I don't know if DirectAdmin tally appreciate it if these files are missing, or maybe it will create new files. I also have some servers where I don't need the bandwidht tally and where I would like to disable the bytes logs, this reduces the iops and give slightly better webserver performance. For better performance there is a "buffer" option for the access_log directive of Nginx which might be used for the bytes logs.

I'm testing automatic cleaning of .bytes.* files with logrotate:

# cat /etc/logrotate.d/directadmin-bytes-files
/var/log/nginx/domains/*.bytes.? /var/log/nginx/domains/*.bytes.?? /var/log/httpd/domains/*.bytes.? /var/log/httpd/domains/*.bytes.??{
        rotate 0
        size 0
        missingok
        nomail
}

This removes all .bytes.* files and keep most recent .bytes file. This shouldn't intervene with DA tally and keeps your bytes logs clean. It can also be done in other ways, for example with a cronjob.