You cannot prevent bruteforcing, much like you cannot prevent a DDoS attack. It happens, and is something you must accept and deal with accordingly.
The only possible caveat to that is locking down specific services and/or ports to trusted IP addresses.
In your case, CSF/LFD is doing its job...