Bandwidth usage 78 TB but no proof

kebirhost

Verified User
Joined
Jul 8, 2019
Messages
83
Hello,

Same issue on our servers also. I hope DA team will fix the issue ASAP.

Thanks,
Melih
 

DirectAdmin Support

Administrator
Staff member
Joined
Feb 27, 2003
Messages
8,921
Hello,

Just an update, I've tracked one case down to an openlitespeed bandwidth bug, combined with a recent DA fix to read that bad data, which was previously not able to be read. For examples below, replace RESELLER with the name of the affected Reseller.

  1. If you're running openlitespeed, ensure it is updated to the latest version:
    Code:
    cd /usr/local/directadmin/custombuild
    ./build update
    ./build openlitespeed
  2. When a User is deleted, their used bandwidth is logged into that Reseller so it's still counted during the tally, even without the User:
    Code:
    /usr/local/directadmin/data/users/RESELLER/bandwidth.reseller.tally
  3. DA bug where there was no newline added to the end of that entry, so previous DA's count read the bad data, which was hiding the true OLS bw error.
  4. Recent DA fix is now able to read the lines that don't end in a newline, thus including the bogus data in the Reseller's bw count, exposing the issue.

So if think you're affected, check the bandwidth.reseller.tally file for the given Reseller.
If the very first number shows too high, just delete that line (it's a long line with a lot of encoded data).. or if you're unsure, delete the whole contents of the file, then recompute the tally for this Reseller
Code:
cd /usr/local/directadmin
echo 'action=bandwidthtally&value=RESELLER&type=reseller' >> data/task.queue; ./dataskq d432
where debug 432 is the tally debug level for a bit more info.
I've pushed some pre-release binaries which include extra debug output at this level (Compile time >= Oct 23 2019 at 10:56:28), where you'd be looking near the end for Tally::tallyReseller output, and the trace I found mentioned the call to getTallyTotal, which, in this case, is loading in the bandwidth.reseller.tally file.

If you're looking for a quick way to knock the bandwidth into a normal range:
Code:
cd /usr/local/directadmin
rm -f data/users/*/bandwidth.reseller.tally
echo 'action=bandwidthtally&value=all' >> data/task.queue; ./dataskq d432
as long as you're ok with not including bandwidth that belonged to deleted Users.

NOTE: It would also affect other webservers (apache/nginx/litespeed, etc) where the amount is smaller/correct, as the deleted User bandwidth wouldn't have showed up until the #3 fix from 1.53.0. In that scenario, the jump in bandwidth is actually correct, and was should have been shown in the past. Either way, please check the bandwidth.reseller.tally for the affected Reseller to see what's going on.

We've added a new row when an Admin is viewing a Reseller, such that you can now see how much of the total bandwidth is used by the deleted User bandwidth:
https://directadmin.com/features.php?id=2564 (pre-release binaries are now available)

I'll continue to step through the reported cases to see if there is any more information on this and will reply back once I know more.

Sorry for the inconvenience.

John
 

Freddy

Verified User
Joined
Apr 14, 2016
Messages
41
We are experiencing the same problem. We do NOT run openlitespeed. This is an Apache 2.4 webserver. Our reseller bandwidth shows 16 thousand TB!!!
Look at the screenshot. https://i.imgur.com/Ny3shDm.png
There is also something wrong with the reseller statistics information screen because 16000 TB is not infinite. Maybe DirectAdmin does not support PB (petabyte) formatting?

There is no evidence that there was any sort of abnormal bandwidth activity on our host platform. There are also no users that have generated any abnormal amount of bandwidth.

We were running DA version 1.593 at the moment of the problem. I have updated to version 1.594 in the meantime.

[update] I have removed the incorrect line from bandwidth.reseller.tally and recalculated the tally for the given reseller. The bandwidth now shows a normal value.

Before deletion
Code:
Tally::tallyReseller:RESELLER: += 8.20 = 94466.41 from self RESELLER
Tally::tallyReseller:RESELLER: += getTallyTotal(./data/users/RESELLER/bandwidth.reseller.tally):17012397279.00 = 17012491745.41
Tally::tallyReseller:RESELLER: reseller.usage badnwdith=17012491745 has been set from 17012491745.41
Tally::tallyReseller:RESELLER: set deleted_user_bandwidth=17012397279 from 17012397279.00
After deletion
Code:
Tally::tallyReseller:RESELLER: += 8.20 = 94468.56 from self RESELLER
Tally::tallyReseller:RESELLER: += getTallyTotal(./data/users/RESELLER/bandwidth.reseller.tally):0.00 = 94468.56
Tally::tallyReseller:RESELLER: reseller.usage badnwdith=94468 has been set from 94468.56
Tally::tallyReseller:RESELLER: set deleted_user_bandwidth=0.0000 from 0.00
My admin bandwidth still shows the wrong value!
 
Last edited:

Jksan

New member
Joined
Oct 29, 2019
Messages
1
We experienced the same problem. Yesterday A reseller deleted a user, and his bandwidth was calculated as 2710.07% of his usual 10TB bandwidth. The reseller and his users were suspended and unhappy.

We are using Apache 2.4.41 and DA 1.594. I used the 3 line quick fix to solve the problem for now. Actual bandwidth usage for the reseller is 17GB.
 

loekg

New member
Joined
Jan 30, 2015
Messages
1
212TB traffic

I've experienced the same issue yesterday (nginx_apaches setup) where a reseller had used up 212TB traffic. This server is capped at a line speed of 500mbit which would require 49 days of non-stop 100% line saturation to use up this amount of traffic, needless to say this is physically impossible.
 

DirectAdmin Support

Administrator
Staff member
Joined
Feb 27, 2003
Messages
8,921
Check the mentioned bandwidth.reseller.tally file, and if you're not sure, please create a ticket.

John
 

indositehost

New member
Joined
Jun 30, 2019
Messages
5
We experienced the same problem. Yesterday A reseller deleted a user, and his bandwidth was calculated as 2710.07% of his usual 10TB bandwidth. The reseller and his users were suspended and unhappy.

We are using Apache 2.4.41 and DA 1.594. I used the 3 line quick fix to solve the problem for now. Actual bandwidth usage for the reseller is 17GB.
How and what is the 3 line quick fix to solve the problem?
 

indositehost

New member
Joined
Jun 30, 2019
Messages
5
on 28 Oct 2019 I face again same issue

https://i.imgur.com/Cy1oK35.png

with latest DA
Yes me too. i have faced this issue at Nov 3rd 2019 on DA version 1.59.4 on Apache server + php-fpm only.

So fix this and do this:

So if think you're affected, check the bandwidth.reseller.tally file for the given Reseller.
If the very first number shows too high, just delete that line (it's a long line with a lot of encoded data).. or if you're unsure, delete the whole contents of the file, then recompute the tally for this Reseller


Code:
cd /usr/local/directadmin
echo 'action=bandwidthtally&value=RESELLER&type=reseller' >> data/task.queue; ./dataskq d432
Notes above code: RESELLER replaced by username of the reseller
 

rotame

Verified User
Joined
Jul 4, 2019
Messages
50
Hi,

The same for me suspended all accounts

WEB SETUP Nginx + Apache
php-fpm
Directadmin : version=1.59.4
OS : Centos 7
 

indositehost

New member
Joined
Jun 30, 2019
Messages
5
Hi,

Yes, we are still facing this issue.
This is an automated message notifying Reseller username that 971.468 % of their bandwidth has been used up.


it's look like the bug start counting abnormal tally bandwidth from here:
While a reseller terminate/delete a user and then reCreate add user again.

So this bug should be fixed in next release, please?
 

webquarry

Verified User
Joined
Mar 19, 2004
Messages
177
We are experiencing the same issue. Never have used OLS. these are on apache web servers. Problem appears to be that the deleted user's bandwidth is stored in the bandwidth.reseller.tally file in BYTES but the control panel reads the value as MEGABYTES.
 

DirectAdmin Support

Administrator
Staff member
Joined
Feb 27, 2003
Messages
8,921
Hello all,

More digging, I may have found a better fix.
Please try the pre-release binaries and issue a full tally.
Hopefully that should bring things back in line.

Let me know how it goes, one way or another.

John
 
Top