Hello,
Jeff forwarded me to this thread, I'll just follow up with a few comments about the dataskq in the new version of DA.
The very first time the Brute Force Monitor (BFM) is run, it will scan all logs from the start (eg: /var/log/messages, /var/log/secure, /var/log/maillog, etc... varying per OS). So if your logs don't get rotated often, are large, and stretch back a long way, then the dataskq of the BFM will have to parse a decent amount of data on the first run. Subsequent runs should be ok after that.
In 1.39.3 there are significant optimizations with regards to the BFM logging (brute_log_entries.list), but if the number of attempts is excessively high for the full span of the log, they can become rather oversized (including brute_log_entries.list, brute_ip.data and brute_user.data).
A few easy things can be done to help minimize the load on the box for these runs, as well as a few tips/pointers on the system.
1) Rotate your system logs more often. Eg, in /etc/logrotate.conf, change "weekly" to be "daily"... that way, the first run won't be as hard.
2) In 1.39.2+, the optimizations will mean that the dataskq (for any subsequent runs) should essentially have minimal load (assuming your brute_ip.data and brute_user.data don't get too large). The clearning of the logs are in with the tally now, so running the tally should dump out old values. Keeping the time settings "short" (for time to keep logs) will help keep them smaller... but once things are running, it shouldn't matter. Again, it's the first run which spans too far back which can cause the issues.
3) This might be an option for a future release, but we could make the first run simply start at the end of the current log files, and scan from there as they fill. This would mean any old attacks in the past would not be seen, but the logging would start from the time it was turned on (no backtracking), so the load wouldn't spike on the first run. Let me know if anyone would want that option.
4) Regarding the error.log message
Code:
ServerInfo::fill_brute_force_tokens : fill tables
, that would be due to a very large brute_log_entries.list file. Any easy way to shrink it (if you don't want to go all the way back) is to type:
Code:
cd /usr/local/directadmin/data/admin
mv brute_log_entries.list brute_log_entries.list.backup
tail -n 1000 brute_log_entries.list.backup > brute_log_entries.list
and your brute_log_entries.list file is now only 1000 lines long, giving DA a better change to process it all more quickly. Once your nightly tally dumps out old values from it (after "x" number of days after the entries were found), that would also help. Note that DA stamps the time the entry was found.. not the time the entry was logged. So the first run, spanning back however many weeks, will all be stamped with the current time... hence you'd need to wait x number of days from the time DA noticed it, not the logged date in the entry.
5) You can also delete the brute_user.data file if it's got too many entries (again, due to the large time span). You could also delete brute_ip.data, but that would defeat the point, since it's what logs who did what. It's not usually large anyway, since BF attacks tend to come from single IPs. (but the usernames can vary, hence the brute_user.data can be larger)
Anyway, for 1.39.2+, it's disabled for existing installs during the update, and only enabled by default for new installs.
Ensure it's set as desired after updating.
John