Stats Control (Yet another AwStats Plugin ;)

canda · Apr 22, 2006

For the benefit of those that are following this thread I thought I would post my findings on this.

I just tried this on Debian, here is the output of installing the plugin (I had only webalizer before):

Stats_Control successfully downloaded
plugin.tar.gz extracted
install.sh has been run:

chown: `root:wheel': invalid group
+ cp -R /usr/local/directadmin/plugins/Stats_Control/scripts/.plugins/stats_control /home/admin/.plugins/
+ chown -hR admin:admin /home/admin/.plugins
+ cp /usr/local/directadmin/plugins/Stats_Control/scripts/awstats.conf /etc/awstats/awstats.conf
+ ln -fs /usr/local/bin/webalizer /usr/local/bin/webalizer_default
+ cp -R /usr/local/directadmin/plugins/Stats_Control/awstats/icons/browser /usr/local/directadmin/plugins/Stats_Control/awstats/icons/clock /usr/local/directadmin/plugins/Stats_Control/awstats/icons/cpu /usr/local/directadmin/plugins/Stats_Control/awstats/icons/flags /usr/local/directadmin/plugins/Stats_Control/awstats/icons/mime /usr/local/directadmin/plugins/Stats_Control/awstats/icons/os /usr/local/directadmin/plugins/Stats_Control/awstats/icons/other /var/www/html/awstatsicons/
+ perl -pi -e 's/(webalizer|rotation)=1/${1}=0/' /usr/local/directadmin/conf/directadmin.conf
+ echo -e '40 0 * * * root /usr/local/directadmin/plugins/Stats_Control/scripts/stats_run.sh #Added by Stats Control Plugin'
+ set +xe
/usr/local/directadmin/plugins/Stats_Control/scripts/install.sh: line 40: service: command not found
Plugin Installed!

So aside from not having a group as used for ownership and debian not knowing what command "service" is (should this be there, I've encountered this on several scripts I've tried from these forums?) everything appears to be working ok.

I manually ran the update script after configuring it on one site and it worked ok, albeit with only stats showing for today. Is this to be expected?

Also it's not resolving IPs, which conf file should I add DNSLookup=1 into?

canda · Apr 24, 2006

I spoke too soon, I've had the same permission problems when creating static pages but I manually fixed this.

However it's not being updated via the crontab, it's in there but not being run. Did anyone find a reason for this?
Manually running stats_run.sh works fine.

IT_Architect · Apr 25, 2006

ClayRabbit, I have some questions.

Logs rotated monthly

That sounds highly impractical. A single server generates 80 to 100 meg. log files a day. Few people would wait a month to know their stats. Can that be changed back to daily?

During install, this plugin will disable DirectAdmin's log rotation and webalizer run (and will enable those features after uninstall).

What happens to DA’s bandwidth tracking while this plug-in is in control of the log rotation?

Chrysalis · Apr 25, 2006

Installed and no errors popped up, have enabled for a domain set to 1st of month for rotation.

I am a bit unsure of how DA normally works with log rotation etc.

But with it been monthly does this mean the tallies are also done monthly for traffic or they still get done daily as before?

Should the existing DA crontab entries remain in place? mine have.

The awstats viewer I disabled and set make a static page so this page be same as fusion's plugin? and when I click the link to the page it is invalid but I assume this is because it hasnt created a page yet.

I share IT_Architect concerns regarding the log data, on fusion's plugin the only way the stats could be generated on a cron without losing data was to rotate the logs and generate the stats in the same crontab command, is there no loss of stats as a result of the plugin pulling the stats before DA rotates so the gap between the 2 is unstatted data or am I running up the wrong tree here?

I also have concerns that awstats will be a crunching a month's worth of numbers, currently the servers I run awstats on take around 6-18 hours to compile 24 hours worth of stats and I dont like the idea of having awstats running for many days crunching numbers.

genexis · Apr 26, 2006

why can't we just edit the crontab timing to change it to one day instead of 1 month ? From what i understand awstats uses a command line to update its stats, which is timed by cron.

Sven · Apr 26, 2006

FeBox said:

Yep....here it is....

Code:

#!/bin/bash ############################################################ # AWStats converter for DirectAdmin # # # # Converts the awstats directories that were previously # # used for fusionictnl's plugin, and gets them ready for # # ClayRabbit's plugin. # # # # Coded by Anthony Drimones of FeBox, LLC. # # Contact: anthony [@] febox.com # ############################################################ find /home/*/*/*/public_html/ -name awstats > dirlist echo "Directory List Compiled" for d in `cat dirlist`; do ln -sf /usr/local/directadmin/plugins/Stats_Control/awstats/cgi-bin/lang/ $d/lang ln -sf /usr/local/directadmin/plugins/Stats_Control/awstats/cgi-bin/lib/ $d/lib ln -sf /usr/local/directadmin/plugins/Stats_Control/awstats/cgi-bin/plugins/ $d/plugins rm $d/icon rm $d/awstats.pl cd $d echo '#!/usr/bin/perl' > awstats.pl echo "require '/usr/local/directadmin/plugins/Stats_Control/awstats/cgi-bin/awstats.pl';" >> awstats.pl perms=`echo $d | cut -f3 -d'/'` chown $perms.$perms awstats.pl chmod 755 awstats.pl cd / done echo "Conversion complete!"

I had the plugin of fusionictnl installed and ran this converter script after I installed the Stats Control plugin (25/04/2006).
After I ran the converter script I wanted to remove my old Awstats plugin, so pushed the remove button in stead of the un-install button.

When I enter my old stats (/awstats) I get the following:

Code:

Error: Access to statistics is only allowed from an authenticated session to authenticated users.

Setup ('/etc/awstats/awstats.conf' file, web server or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs' directory).

So I uploaded the old Awstats plugin without installing it and pushed the un-install button:

Code:

uninstall.sh has been run:

Plugin Un-Installed!

But I still get the above error when I try to get to my old awstats.

Another fact:
When I go to the new awstats (cgi-bin/awstats/awstats.pl?...)
I only have the stats from 24/04/2006 to 26/04/2006.
So the converter didn't work

I just want to totally remove the old awstats plugin.

canda · Apr 26, 2006

Does anyone know what permissions get changed and on what files when the update is run?

I'm getting the same problem as everyone else, first time installation works fine and creates the static pages, any subsequent update and I get the same permissions message as everyone else and the information isn't updated when viewing via awstats.pl.

Incidentally I tried updating the plugin and this seemed to reset the permissions to how they should be, it ran fine for the first time and then failed subsequently, again not updating information viewed via awstats.pl.

This is all via command line, the cron doesn't run.

ClayRabbit · Apr 26, 2006

canda said:
However it's not being updated via the crontab, it's in there but not being run. Did anyone find a reason for this?
Manually running stats_run.sh works fine.

If command is added to crontab but not running (regarding to /var/log/cron entries) then it's cron problem...

IT_Architect said:
That sounds highly impractical. A single server generates 80 to 100 meg. log files a day. Few people would wait a month to know their stats.

Why? Logs is constantly accessible without rotation. In my opinion, "impractical" - is hundreeds of tarballs inside logs directory.

Can that be changed back to daily?

Currently I'm not planning to do that.

What happens to DA’s bandwidth tracking while this plug-in is in control of the log rotation?

Nothing. BW counted with *.bytes logfiles - they are completely unaffected by our plugin.

Chrysalis said:
But with it been monthly does this mean the tallies are also done monthly for traffic or they still get done daily as before?

No. Of course BW tally runs as always.

Should the existing DA crontab entries remain in place? mine have.

Yes. Tally runs daily, we just disabling log rotation and webalizer start during it's run.

The awstats viewer I disabled and set make a static page so this page be same as fusion's plugin? and when I click the link to the page it is invalid but I assume this is because it hasnt created a page yet.

"AwStats viewer" is tha same view as in fusion's plugin - via cgi-script. Static pages is like webalizer pages generation.

I share IT_Architect concerns regarding the log data, on fusion's plugin the only way the stats could be generated on a cron without losing data was to rotate the logs and generate the stats in the same crontab command, is there no loss of stats as a result of the plugin pulling the stats before DA rotates so the gap between the 2 is unstatted data or am I running up the wrong tree here?

Sorry, I don't understand question. Of course the only way to generate stats without loosing data is to rotate logs with same scritp which handles stats run.

genexis said:
why can't we just edit the crontab timing to change it to one day instead of 1 month ?

To change what? Stats generation runs every day.

ClayRabbit · Apr 26, 2006

canda said:
Does anyone know what permissions get changed and on what files when the update is run?

Currently, commands

Code:

    chown -h :$USER "$CONFIG" && chmod 640 "$CONFIG"
    if [ -d "$DOM_DIR/stats" ]; then chown -hR $USER:$USER "$DOM_DIR/stats/"; fi

is executed when config file for domain created for a first time and every time when it's updated (that's when user stats config or one of config templates is updated).
If stats directory and files inside it becomes owned by root again then it seems like DirectAdmin still running webalizer every day. Make sure you have "webalizer=0" inside your directadmin.conf.
BTW, have you "rotation=0" inside directadmin.conf also?

IT_Architect · Apr 26, 2006

Why? Logs is constantly accessible without rotation. In my opinion, "impractical" - is hundreeds of tarballs inside logs directory.

I'm sure you understand this area better than others here so help us to understand. A 3-5 GIG log file is OK? What does that do to the duration of the stats runs as you get closer to the end of the month? It may not need to do lookups on everything but wouldn't it need to either skip through all of the previously processed rows rows to get to where it needed to start processing the new stats or go through the directory extents to locate the end of the file and begin reading backwards? What are we missing? When it finally does get tarred off, I would guess a file that size would take a good while and a lot of CPU to tar. I assume that you would do a rename on the file so that the a new log could start right away while the old one is being tarred?

It seems like there are a lot of people on this thread who have installed the plug-in with varying degrees of success, but I don't see any getting stats. So all or nearly all of us must be missing something. We need a little better understand of your plugin I think before we can use it as you intended.

ClayRabbit · Apr 26, 2006

IT_Architect said:
A 3-5 GIG log file is OK?

We never have such log files on our servers. However plugin have additional option where you can specify maximum log size, so it will be rotated any day when it's goes over specified limit. Plugin works that way just because monthly rotation suits OUR needs and neeeds of OUR clients better.

I assume that you would do a rename on the file so that the a new log could start right away while the old one is being tarred?

Yes, of course script doing that during rotation.

Chrysalis · Apr 27, 2006

ok thanks I understand now, so stats generation is daily the tallying remains daily and you changed log archiving to monthly or when specified size is hit, sounds good to me.

I checked my configuration again and its set like this for the user.

Statistic reports configuration - use awstats

when clicking awstats configure

Open details in new window - always
Build static pages - yes (was no when last stats generated)

awstats viewer

Use frames - yes

now when I click view stats (shortcut to index.html) it shows webalizer

when I click View stats with AWStats viewer (shortcut to awstats.pl) it takes me to a invalid url on the homepage of the site.

So I navigated to public_html/cgi-bin/awstats dir in the DA file manager the files appear to be in place but even when navigating there in the browser it just redirects to a bad url on the site.

So I then tried a http://ip/~username/cgi-bin/awstats/ url and that worked but then I get the following error.

Error: awstats failed to create directory DirData (DirData="/home/admin/domains/sharedip/../stats", CreateDirDataIfNotExists=1).

Setup ('/etc/awstats/awstats.conf' file, web server or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs' directory).

Now this probably is caused because /home/admin/domains isnt writeable by the script so I created the dir in the shell and then the page loads.

The stats are 0 and thats where I am at now.

It would seem for whatever reason at least on this domain (runs postnuke index.php in public_html) that trying to load awstats via the vhost fails postnuke throws a invalid module error, if loading via userdir it works but then new problems occur with file access permissions.

Is it possible to have the plugin use ip based url's to avoid conflicts with some sites? if yes what needs to be done to get it working with stats.

If no then how could it possibly be made to stop conflicting with existing sites.

--edit--

I know why I got 0 stats now.

Statistics for:
(ip address)

so it loads the stats dynamically based on the url accessed from.

ClayRabbit · Apr 27, 2006

DirData and Host is determined dynamically only while individual config file for domain specified in "config=" variable (inside url) not exists in /etc/awstats.
Domain config file is created during first stats run after awstats was enabled in control panel of that user. After that there is no matter how script is accessed - it's always using existing config specified in "config=" variable.

canda · Apr 27, 2006

ClayRabbit said:
If stats directory and files inside it becomes owned by root again then it seems like DirectAdmin still running webalizer every day. Make sure you have "webalizer=0" inside your directadmin.conf.
BTW, have you "rotation=0" inside directadmin.conf also? [/B]

ClayRabbit, that was it!

I added webalizer=0 and rotation=0 into directadmin.conf and it is now working perfectly, all my sites set with awstats were updated last night at 00:40

Thanks!

canda · Apr 28, 2006

ClayRabbit

As per my previous post, awstats is now working perfectly (I've even added a plugin for whois lookups) and its updating nightly.

However webalizer is not working on the sites where I have used your plugin select it rather than awstats.

I get the following message in the webalizer.log in the sites stats directory:

/usr/local/directadmin/plugins/Stats_Control/scripts/webalizer_user.sh: line 20: /usr/local/bin/webalizer_default: No such file or directory

I've checked and /usr/local/bin/webalizer_default is a symlink to /usr/local/bin/webalizer so should work? Or is it a problem with the config.
Any ideas?

ClayRabbit · Apr 28, 2006

Try to run /usr/local/bin/webalizer_default in shell and meke sure it works.

canda · Apr 28, 2006

doh... I didn't actually check to see if it linked to an existing file, which it didn't. Webalizer was in /usr/bin so I changed the link to that and it works fine.

Thanks again

Chrysalis · Apr 28, 2006

after my config changes it seems to be working now.

dan35 · Apr 29, 2006

I got the same errors.

View stats
Error: awstats failed to create directory DirData (DirData="/../stats", CreateDirDataIfNotExists=1).

Setup ('/etc/awstats/awstats.conf' file, web server or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs' directory).

View stats with AWStats viewer
404

ClayRabbit · Apr 30, 2006

For same error try the same solution

Stats Control (Yet another AwStats Plugin ;)

Verified User

Verified User

Verified User

Verified User

Verified User

New member

Verified User

Verified User

Verified User

Verified User

Verified User

Verified User

Verified User

Verified User

Verified User

Verified User

Verified User

Verified User

Verified User

Verified User