Solved Rebuild awstats static .html for previous months

T

twv

Verified User
Joined
Oct 31, 2003
Messages
219
I am moving users from a non-DirectAdmin server to a DA server and trying to copy over their awstats history. I have copied the .txt data files such as awstats012023.DOMAIN.txt to /home/USER/domains/DOMAIN/awstats/.data/

Using this script:

Website statistics | Directadmin Docs

DirectAdmin Knowledge Base
docs.directadmin.com

I get this error (domain and user obfuscated):
Code: 
Launch update process : "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=DOMAIN -update -configdir=/home/USER/domains/DOMAIN/awstats/.data
Create/Update database for config "/home/USER/domains/DOMAIN/awstats/.data/awstats.DOMAIN.conf" by AWStats version 7.9 (build 20230108)
From data in log file "/var/log/user_logs/USER/DOMAIN.log"...
Error: Couldn't open server log file "/var/log/user_logs/USER/DOMAIN.log" : No such file or directory
Setup ('/home/USER/domains/DOMAIN/awstats/.data/awstats.DOMAIN.conf' file, web server or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs' directory).

/var/log/user_logs/ is in fact empty (nothing in it at all), but that is where DA’s awstats.conf file points:
Code: 
LogFile="/var/log/user_logs/USER/DOMAIN.log"

Can anyone help?
 
twv said:
but that is where DA’s awstats.conf file points:
Click to expand...
DA does not have an awstats.conf file at least I can't find that on any of my servers.
DA has seperate awstats.www.domain.conf files symlinked to awstats.domain.conf file in the same directory.
Indeed that's is pointing to the location you say, but that only contains a .readme.txt file as this directory is only used for processing.
twv said:
Setup ('/home/USER/domains/DOMAIN/awstats/.data/awstats.DOMAIN.conf' file, web server or permissions) may be wrong.
Click to expand...
I presume the .data directory is chown to user:user correct?

The conf file should look like this:
Code: 
-rw-r--r-- 1 user user  64K 2024-03-05 00:14 awstats.userdomain.com.conf
lrwxrwxrwx 1 user user   29 2023-08-23 00:37 awstats.www.userdomain.com.conf -> awstats.userdomain.com.conf

If both permissions and owner are correct, then this part is good.
 
  • Like
Reactions: twv
Richard G said:
DA does not have an awstats.conf file at least I can't find that on any of my servers.
Click to expand...
You’re right, sorry -- sloppy on my part. I meant awstats.DOMAIN.conf.

I presume the .data directory is chown to user:user correct?
Click to expand...
Correct.

If both permissions and owner are correct, then this part is good.
Click to expand...
Yes, it all looks right:
Code: 
ls -la /home/USER/domains/DOMAIN/awstats/.data/
total 5688
drwxr-xr-x 2 USER USER   4096 Mar  5 12:56 .
drwxr-xr-x 3 USER USER   4096 Mar  5 12:54 .. 
-rw-r--r-- 1 USER USER 134154 Jan 31  2023 awstats012023.DOMAIN.bak
-rw-r--r-- 1 USER USER 134145 Feb  1  2023 awstats012023.DOMAIN.txt
-rw-r--r-- 1 USER USER 245718 Feb  1 00:01 awstats012024.DOMAIN.bak
-rw-r--r-- 1 USER USER 245547 Feb  1 01:01 awstats012024.DOMAIN.txt
-rw-r--r-- 1 USER USER 145199 Mar  1  2023 awstats022023.DOMAIN.bak
-rw-r--r-- 1 USER USER 145173 Mar  1  2023 awstats022023.DOMAIN.txt
-rw-r--r-- 1 USER USER 153223 Feb 29 11:01 awstats022024.DOMAIN.bak
-rw-r--r-- 1 USER USER 154108 Feb 29 12:01 awstats022024.DOMAIN.txt
-rw-r--r-- 1 USER USER  96314 Apr  1  2023 awstats032023.DOMAIN.bak
-rw-r--r-- 1 USER USER  96220 Apr  1  2023 awstats032023.DOMAIN.txt
-rw-r--r-- 1 USER USER  64233 Mar  5 00:18 awstats032024.DOMAIN.txt
-rw-r--r-- 1 USER USER  13095 Apr 30  2022 awstats042022.DOMAIN.bak
-rw-r--r-- 1 USER USER  13065 May  1  2022 awstats042022.DOMAIN.txt
-rw-r--r-- 1 USER USER 112930 May  1  2023 awstats042023.DOMAIN.bak
-rw-r--r-- 1 USER USER 112921 May  1  2023 awstats042023.DOMAIN.txt
-rw-r--r-- 1 USER USER  55400 Jun  1  2022 awstats052022.DOMAIN.bak
-rw-r--r-- 1 USER USER  55361 Jun  1  2022 awstats052022.DOMAIN.txt
-rw-r--r-- 1 USER USER 144529 Jun  1  2023 awstats052023.DOMAIN.bak
-rw-r--r-- 1 USER USER 144387 Jun  1  2023 awstats052023.DOMAIN.txt
-rw-r--r-- 1 USER USER  85724 Jul  1  2022 awstats062022.DOMAIN.bak
-rw-r--r-- 1 USER USER  85714 Jul  1  2022 awstats062022.DOMAIN.txt
-rw-r--r-- 1 USER USER 117419 Jul  1  2023 awstats062023.DOMAIN.bak
-rw-r--r-- 1 USER USER 117317 Jul  1  2023 awstats062023.DOMAIN.txt
-rw-r--r-- 1 USER USER  73305 Aug  1  2022 awstats072022.DOMAIN.bak
-rw-r--r-- 1 USER USER  73295 Aug  1  2022 awstats072022.DOMAIN.txt
-rw-r--r-- 1 USER USER 164768 Aug  1  2023 awstats072023.DOMAIN.bak
-rw-r--r-- 1 USER USER 164730 Aug  1  2023 awstats072023.DOMAIN.txt
-rw-r--r-- 1 USER USER  76619 Sep  1  2022 awstats082022.DOMAIN.bak
-rw-r--r-- 1 USER USER  76609 Sep  1  2022 awstats082022.DOMAIN.txt
-rw-r--r-- 1 USER USER 173989 Sep  1  2023 awstats082023.DOMAIN.bak
-rw-r--r-- 1 USER USER 173754 Sep  1  2023 awstats082023.DOMAIN.txt
-rw-r--r-- 1 USER USER  77557 Oct  1  2022 awstats092022.DOMAIN.bak
-rw-r--r-- 1 USER USER  77373 Oct  1  2022 awstats092022.DOMAIN.txt
-rw-r--r-- 1 USER USER 152921 Sep 30 23:01 awstats092023.DOMAIN.bak
-rw-r--r-- 1 USER USER 152886 Oct  1 00:00 awstats092023.DOMAIN.txt
-rw-r--r-- 1 USER USER  98709 Nov  1  2022 awstats102022.DOMAIN.bak
-rw-r--r-- 1 USER USER  98658 Nov  1  2022 awstats102022.DOMAIN.txt
-rw-r--r-- 1 USER USER 134887 Oct 31 23:01 awstats102023.DOMAIN.bak
-rw-r--r-- 1 USER USER 134770 Nov  1 00:01 awstats102023.DOMAIN.txt
-rw-r--r-- 1 USER USER 114845 Dec  1  2022 awstats112022.DOMAIN.bak
-rw-r--r-- 1 USER USER 114738 Dec  1  2022 awstats112022.DOMAIN.txt
-rw-r--r-- 1 USER USER 174886 Dec  1 00:01 awstats112023.DOMAIN.bak
-rw-r--r-- 1 USER USER 174643 Dec  1 01:01 awstats112023.DOMAIN.txt
-rw-r--r-- 1 USER USER 103712 Dec 31  2022 awstats122022.DOMAIN.bak
-rw-r--r-- 1 USER USER 103732 Jan  1  2023 awstats122022.DOMAIN.txt
-rw-r--r-- 1 USER USER 150291 Jan  1 00:01 awstats122023.DOMAIN.bak
-rw-r--r-- 1 USER USER 149945 Jan  1 01:01 awstats122023.DOMAIN.txt
-rw-r--r-- 1 USER USER  64580 Mar  5 00:18 awstats.DOMAIN.conf
lrwxrwxrwx 1 USER USER     27 Mar  1 00:21 awstats.www.DOMAIN.conf -> awstats.DOMAIN.conf

I copied the awstats#######.DOMAIN.txt and .bak files from the previous server.

Did you use this script to try and rebuild?
Click to expand...
Yes, and it gave the error I posted above: Couldn't open server log file "/var/log/user_logs/USER/DOMAIN.log" : No such file or directory

This is what the DA page says: "Assuming the data for the required months does exist in /home/username/domains/domain.com/awstats/.data, you should be able to regenerate your static HTML pages for those months. The script below can do it for you."
 
twv said:
Couldn't open server log file "/var/log/user_logs/USER/DOMAIN.log" : No such file or directory
Click to expand...
But at least the /var/log/user_logs directory does exist and contains the .readme.txt file correct?

I'm just wondering, maybe the script can't create that directory itself if no database already exist.
What if you try to create the /user/domain.log file yourself in the /var/log/user_logs directory? Just to be sure chown it to the user and leave it as empty file.
I'm no scripter so I could be wrong, but it's possible it will be filled with data once present. It's at least something worth trying.

However if it's for other users too as you say, this can be a very intensive work.

I presume you have enabled awstats for all users now.
In that case it might be best to wait until the normal awstats cron has run or the user tally, at least that current awstats for today are generated so all domains are in the database. And then retry the script tomorrow.

Or do all sites already have the current awstats present? If yes then something might be missing in the rebuild script.
You could consider sending in a ticket for that.
 
  • Like
Reactions: twv
What if you try to create the /user/domain.log file yourself in the /var/log/user_logs directory? Just to be sure chown it to the user and leave it as empty file.
Click to expand...
You’re brilliant! That worked.

I finally read the .readme.txt file, which says "This folder is for temporary http log hard-links or copies, for awstats processing as the User.\nIt should usually be empty, less this file, unless awstats is running for a domain."

So I guess I will remove /var/log/user_logs/user/ and /var/log/user_logs/user/domain.log once I have run the script for all the months.
 
Shouldn't that script be updated to create those files? For my future self:

Bash: 
#!/bin/sh
if [ "$#" -eq 0 ]; then
         echo "Usage:";
         echo "    $0 <MM> <YY>";
         exit 1;
fi

month=$1
short_year=$2
full_year=20${short_year}

for u in `ls /usr/local/directadmin/data/users`; do
#Create tmp dir
mkdir /var/log/user_logs/$u
chown $u /var/log/user_logs/$u
{
         for d in `cat /usr/local/directadmin/data/users/$u/domains.list`; do
         #Create tmp logfile
         touch /var/log/user_logs/$u/$d.log
         chown $u /var/log/user_logs/$u/$d.log
         {
                   echo "";
                   echo "$u: $d: $month $full_year";
                   DATA=/home/$u/domains/$d/awstats/.data/awstats${month}${full_year}.${d}.txt
                   if [ ! -s $DATA ]; then
                             echo "Cannot find $DATA for $month $full_year. Skipping.";
                             continue;
                   fi

                   /usr/bin/perl /usr/local/awstats/tools/awstats_buildstaticpages.pl \
                       -config=$d -configdir=/home/$u/domains/$d/awstats/.data -update \
                       -diricons=icon -awstatsprog=/usr/local/awstats/cgi-bin/awstats.pl \
                       -dir=/home/$u/domains/$d/awstats -builddate=${short_year}${month} \
                       -year=$full_year -month=$month

                   echo "";
         }
         done;
}
#Remove tmp dir
if [ -n "$u" ]; then
rm -r /var/log/user_logs/$u
fi
done;
exit 0;
 
Last edited:
  • Like
Reactions: twv
You must log in or register to reply here.
Back
Top