Rotate logs more than once per day

A

Ack

New member
Joined
Oct 2, 2005
Messages
2
I have a very high traffic website running on CentOS and using Directadmin. The site generates more than 2 GB per day of httpd logs, and when the log file size reaches 2 GB Apache slows way down and gets all screwy. I've been manually logging in, deleting the log file, and restarting httpd every time the site slows down.

Is there any way to get Directadmin to rotate the httpd log for this site 2 times per day (and possibly 3 times per day if the traffic continues to grow)?

If not, can you tell me how to disable the log rotating that directadmin does and manually set something up myself to rotate or delete the log twice per day without breaking anything?

Thanks for any help you can provide!
 
Thanks! So when it runs the tally will it delete all of the old bloated httpd log files?
 
rotate logs

Hi,

I'm having the same problem. Did you ever figure out if the tally command also rotates the log files?

I've got a log file: var/log/httpd/domains/mydomain.com.log that is reaching 2GB every day and causing Apache to crap out on me. So far my only solution is:

rm mydomain.com.log
touch mydomain.com.log
restart apache

I was also trying to get things to rotate at a different time of day so I changed the line in the cron file from:

10 0 * * * root echo 'action=tally&value=all' >> /usr/local/directadmin/data/task.queue

to:

10 6 * * * root echo 'action=tally&value=all' >> /usr/local/directadmin/data/task.queue

But it didn't seem to do anything.

I've also been reading about logrotate, but I don't seem to understand it, and I can't find anything that deals with the domain level files, just the files in the directory above.

Any pointers would be greatly appreciated!

Thanks,
Rebecca
 
OK... did some fiddling and confirmed that running the tally does indeed rotate the log files.

Now I Just need to make sure the cron file is working correctly.
 
Re: rotate logs

Rebecca said:
... cron file from:

10 0 * * * root echo 'action=tally&value=all' >> /usr/local/directadmin/data/task.queue

to:

10 6 * * * root echo 'action=tally&value=all' >> /usr/local/directadmin/data/task.queue

But it didn't seem to do anything.
Click to expand...

You change tally time from 00:10hr everyday to 06:10hr everyday

if you want to rotate twice per day you need to replaced with this

// this will retate every 12 hr at 0 and 12
10 0/12 * * * root echo 'action=tally&value=all' >> /usr/local/directadmin/data/task.queue

// this will retate every 12 hr at 6 and 18
10 6/12 * * * root echo 'action=tally&value=all' >> /usr/local/directadmin/data/task.queue

// or use two line of specific time
10 3 * * * root echo 'action=tally&value=all' >> /usr/local/directadmin/data/task.queue
10 14 * * * root echo 'action=tally&value=all' >> /usr/local/directadmin/data/task.queue
// this will rotate at 03:10hr and 14:10hr

Hope this help
 
can't get changes to take

Hmmm...

I'm still having problems. I can't seem to get changes to the directadmin_cron file to work. I can change the file, but it doesn't seem to trigger the tally command.

If I run the tally command manually:

echo 'action=tally&value=all' >> /usr/local/directadmin/data/task.queue

then it runs the tally and rotates the logs. But it doesn't seem to run them on the schedule in directadmin_cron.

at 15:00 I edited the directadmin_cron file and set the line to this:

10 15 * * * root echo 'action=tally&value=all' >> /usr/local/directadmin/data/task.queue

Then I saved the file and ran this:

killall -HUP crond

I thought that should make the tally run at 15:10. But the tally didn't run. I waited for over an hour and still it didn't run.

What am I doing wrong?

Thanks,
Rebecca
 
Thanks,

OK... I followed all of the suggestions in the link, couldn't get the tally to run. Here's what I found:

1) /usr/local/directadmin/data/task.queue - File does not exist

2) /var/log/cron appears to show that dataskq is running every minute:

Jul 12 16:50:01 cp crond[20833]: (root) CMD (/usr/local/directadmin/dataskq)
Jul 12 16:51:01 cp crond[21171]: (root) CMD (/usr/local/directadmin/dataskq)
Jul 12 16:52:01 cp crond[21173]: (root) CMD (/usr/local/directadmin/dataskq)
Jul 12 16:53:01 cp crond[21176]: (root) CMD (/usr/local/directadmin/dataskq)
Jul 12 16:54:01 cp crond[21203]: (root) CMD (/usr/local/directadmin/dataskq)

3) ps ax | grep cron - returns the following:
12587 ? Ss 0:00 crond
15206 pts/0 R+ 0:00 grep cron

4) tried re-starting cron. Don't see any error messages but in the var/log/messages file there are TONS of lines that look like this:

Jul 13 00:29:01 cp crond(pam_unix)[24270]: session opened for user root by (uid=0)
Jul 13 00:29:02 cp crond(pam_unix)[24270]: session closed for user root
Jul 13 00:30:01 cp crond(pam_unix)[24286]: session opened for user root by (uid=0)
Jul 13 00:30:01 cp crond(pam_unix)[24287]: session opened for user root by (uid=0)

5) Tried to run the task.queue manually with:
/usr/local/directadmin/dataskq d

Got this:
In debug mode
Debug set to level 10
root priv set: uid:0 gid:0 euid:0 egid:0
pidfile written
staring queue
done queue

But the tally did not run.

Here is my /etc/cron.d/directadmin_cron file:

* * * * * root /usr/local/directadmin/dataskq
2 0-23/6 * * * root echo 'action=vacation&value=all' >> /usr/local/directadmin/data/task.queue;
#5 5 * * 0 root /sbin/quotaoff -a; /sbin/quotacheck -augm; /sbin/quotaon -a;
50 5 * * * root echo 'action=tally&value=all' >> /usr/local/directadmin/data/task.queue

20 4 1 * * root echo 'action=reset&value=all' >> /usr/local/directadmin/data/task.queue
0 4 * * * root echo 'action=check&value=license' >> /usr/local/directadmin/data/task.queue


Any thoughts are greatly appreciated.
Thanks,
Rebecca
 
Hello,

Try this (running the dataskq doesn't do anything unless there is a task.queue to run):
Code: 
echo "action=tally&value=all" >> /usr/local/directadmin/data/task.queue
/usr/local/directadmin/dataskq d
Also, make sure that:
/usr/local/directadmin/data/admin/admin.list
/usr/local/directadmin/data/admin/reseller.list
Contain all the admins and resellers (respectively) on the system.
Also, for each admin/reseller, make sure that all of their users are listed in:
/usr/local/directadmin/data/users/reseller/users.list

John
 
Hi John,

OK, well if I enter this line of code:

echo "action=tally&value=all" >> /usr/local/directadmin/data/task.queue

The tally will execute. I don't have to enter this line:
/usr/local/directadmin/dataskq d

So I take that to mean that the cron is executing the dataskq properly every minute, and my problem is that somehow my directadmin_cron file is not putting anything into the task.queue. Any way to test that theory?

I checked the files you recommended. I don't have anything in the resellers list, and I don't have a /usr/local/directadmin/data/users/reseller/ directory. But that doesn't surprise me because all I'm doing is serving my own web site which I set up under admin. I never created any resellers or users.

It seems the problem started when I initially made changes to the directadmin_cron file, but I can't be entirely sure because a few changes all happend fairly close together. Here's what's happened over the past few days in case there are any clues here...

1) I installed MRTG
2) I observed that traffic & connections were being truncated
3) I upped maxclients to 1024, which caused traffic and connections to nearly double.
4) I noticed a huge drop in traffic and performance at 00:10 so I figured it was because the logs were being rotated then (I had nuked webalizer ages ago because it was running 24/7 - I put webalizer=0 into the directadmin.conf file... I did not specify rotate=1 because I figured that was the default). So I edited directadmin_cron to run the tally at 06:10 when traffic was light.
4) Then Apache freaked out on me... was still running but wouldn't serve any processes (I have since determined that this was because the log file was over 2gb - but I didn't know that at the time, so I called tech support at my host who did god knows what to the server trying to fix the problem - very bad idea)
5) Then MRTG wouldn't display any graphs, so I un-installed it and re-installed it. I also rebooted the computer. (I later discovered that the technician modified the DocumentRoot in my httpd.conf file which is why the graphs weren't visible).

So, I'm pretty sure that the logs were rotating properly before this whole little adventure because the bandwidth listed in direct admin was updating every night. But I'm not sure which of the above things has caused my problem.

OK, let me know what you think.
Thank you SO much!
Rebecca
 
Try getting it to add the "action=tally&value=all" .. to run ever single minute. You should be able to check your /var/log/cron to see if it's working or not.

It's possible that the pid used to restart cron isn't working. Try a brute force restart of cron:
Code: 
killall -9 crond
/sbin/service crond start
Then check the /var/log/cron and /var/log/messages to see if there are any errors relating to the directadmin_cron file.

John
 
success!

Well, the brute force restart seems to have done the trick!

Thanks so much!
Rebecca
 
a little more information

Just wanted to post this in case anyone else has a similar problem. I was trying to optimize server performance so I was tweaking the directadmin_cron file to change the time the tally ran and encountered the same problem that is described above.

I finally figured out that when I used nano to edit the file, it somehow kept inserting a carriage return in the middle of the tally line. I deleted the carriage return and cron executed perfectly. It now appears that this was the cause of the problem all along, and that the restarting of cron was just a red herring.

Hope this helps somebody out there...
:)
Rebecca
 
You must log in or register to reply here.
Back
Top