Apache off-site log analysis

[caribk]

Implementing an off-site analytics solution for a DA web server w Apache 1.3.34

Somewhat confused about how the log rotation works and log cleanup..
From what I can tell, logs from /var/log/httpd/domains get rotated into /home/user/domains/domain.com/logs/ nightly, then these archives get removed monthly.?

What would be the best practice for copying logs to the off-site location, i.e access logs from /var/log/httpd/domains/ *before* rotation or from the backup directory after rotation?

Also regarding the backup logs, if I were to scp daily access_log files to off-site as access_log.yyyy.mm.dd but still wanted to keep the rotated logs for backup, how does this cleanup work? All old backups from last month removed on 1st of next month?

(Apologize if some of these are Q's I should have answered myself. Not at a location where I can SSH into the DA box and check cron's at time of writing..)

 

[jmstacey]

DirectAdmin's apache log rotation uses the number of backlogs instead of purging them monthly. This value can be set in the adminstrator settings page on the admin level.
But you do have the gist of it, from /var/log to /home/user

Personally, I would write a script that would run about an hour or so after the tally has been run. It would scan all user home directories and ftp the latest logs to my processing server. That way that compressed logs are sent saving bandwidth.

 

[caribk]

thanks jmstacey. good insights. should be piece of cake, just wanted some thoughts of before vs. after rotation log transfer.. for bandwidth purposes that is definately a good point.

 

[bblboy54]

This thread partially is answering the question that I have. Basically, what I need to do is have a script email log files to one of the users on my system every night. So once a day around 1am or so I need to send a file that contains the log files for that user from the previous day (that would end at midnight). I see in the user's log directory for the domain there are log files such as Feb-2008.tar.gz, Feb-2008.1.tar.gz, etc. Here's whats confusing me tho. Feb-2008.tar.gz is dated 02-12, .1 is 02-13, 2 is 2-14, 3 is 2-15 but then Feb-2008.4.tar.gz is dated today and is owned by root. Does the log rotation create the new gz for the day that is beginning and then completes it that night? Also, does the archives continue to increment numbers per day?

If someone could help me grasp how this works a little better, I would really appreciate it

 

[Jinson]

Me too. Wish it would be easier like cpanel's monthly log archive.

 

[tarquel]

Perhaps worth suggesting to DA in the Suggestions section about having more options with what you can do with the logs etc.

Nath.