Sudden problem accessing DA control panel and SSH..

[JGSFla]

Hello,

I own a very large web server in a Tier 3 co-location facility. My server is there for the purpose of hosting one single large web portal I own. I made the mistake of installing DirectAdmin to save money on the Cpanel license and it has been nothing but problems and regrets ever since but this takes the cake..

I had my ssh and the directadmin panel access configured in ports other than the default ones (22 moved to 2425 and 2222 moved to 2324 respectively). It all was working fine until it didn't, for no reason whatsoever. Suddenly I cannot access either, I can access wordpress fine and my site is up but I cannot do administrative stuff at the web server panel level.

I have checked the ports opened and the ip addresses banned log from CSF and there is no particular reason I should be having a problem because the ports show as if they are open and my IP is not banned.

I even asked tech people at my co-location facility to test and see if they had a problem accessing ssh or the control panel, they could not access them just as I couldn't.

My server is currently under a very heavy attack and I just installed and I am configuring Fail2Ban and other measures but that is for the future, I started doing that AFTER I was having the problem with SSH and DA login, in the mean time I have no clue as to how to restore access to the directadmin panel.

I can work on my server using remote management from iLO4 so I am able to do linux changes.

Any ideas would be greatly appreciated.

It is so tiring to have this software that is so important having random problems all the time. If it was not such a huge amount of work I would have moved back to C Panel a while ago, I never had problems with CPanel in over 20 years and hosting thousands of sites. DirectAdmin has been super problematic since the first hour of installing it.

Thanks.

 

[adam12]

Fail2Ban and CSF do the same thing, and will likely conflict and be confusing. I'd recommend sticking to one or the other.

I'd probably eliminate the entire firewall first. Normally I'd suggest disabling CSF but you have Fail2Ban running too, so might be easier to just add a rule that bypasses everything else.

iptables -I INPUT -p tcp --dport 2324 -j ACCEPT

I can't tell from your post if you have SSH working now or not, but you could repeat the command for the SSH port.

 

[exlhost]

Did you try to check if the ports are open with this tool

Offtopic,

Honestly, I don't understand your story. you indicate that you made a mistake moving to directadmin and that inevitably nothing works anymore. It also seems to me that you don't understand anything about it and barely have any knowledge. maybe someone on the forum can help you with paid emergency support.

 

[Richard G]

DirectAdmin has been super problematic since the first hour of installing it.

Most likely because you really have to be a real admin which most cpanel admins are not, as they almost never have to use SSH and do admin things which are not done via the GUI. Because cPanel does everything for you and otherwise cPanel crew can be asked to look.
Because most of us don't have any issues with Directadmin. However that doesn't help you now.

But be aware that if you drive a Ferrari and you want to go drive an Mercedes, it's quite different under the hood and you don't pay Ferrari service and motor. Still Mercedes is a very great car.

First of all, the most important thing you forgot to mention is -why- you can't access Directadmin via SSH or GUI as admin, otherwise said, you didn't post any error notice given. We can't guess what's going wrong then.

Maybe the problems were already caused by the attack, before the attack was really noticable by you. All kinds of options are possible, no need to directly blaim DA.

Anyway, if you can do linux things in another way that is a good start. First of all I would check these:
- In the sshd_config is the port still 2425, in case of emergency, have 22 also open in CSF (both incoming and outgoing)
- Doublecheck this port is still enabled in CSF, test anyway with the default port 22 in case of datacenter blocking of other port for whatever reason
- Check if PermitRootLogin is set to yes (at least so you can login some way, even if it's temporarily)
- Check with iptables -L if all is correct.
- Ofcourse check the logfiles for any error messages when trying to login as admin in DA and as root do SSH. I presume you know what logs to look at.

And if you reply back, then don't forget to mention errors seen or notices when trying to login via SSH and Directadmin GUI as admin.

You already had permission issues in the beginning (click) which better had been fixed then. Might have been caused by installing directadmin and not waiting long enough for the background installation to finish, so I hope you waited long enough. But already with those issues you should have send in a ticket.

You might indeed want to consider to hire some professional help. Also guys from the forum like @zEitEr's and @smtalk's company's can be hired. And Rack911 is also on the partner page.

If you really think DA is to blaim, I would also send in a ticket to DA's ticket system.

 

[JGSFla]

Interesting replies I will try the IP Tables command, I do have access to my machine vie remote desktop in iLO4 so I can bypass the whole SSH idea.
As far as the other comments, lots of smartasses in these places as usual. I am a programmer in 4 computer languages, I have developed over one thousand large and complex websites hundreds of CMS ones, developing lots of my own programmatic solutions (no plugins).

In 1997 before some of the people around here were even born I had already deployed my first apache server from the ground up, when you used to program the whole thing manually, and creating thousands of individual zone files and program manually your httpd.conf files, I was also an expert on IIS at the time. I was managing THEN 5088 websites.

For easier management I started using first Plesk then CPanel, More recently some garbage called Webuzzo, tried recently Cyberpanel as well (that one was absolutely disastrous as it does not work with a linux build it has it sort of built in so getting rid of that required me to re-do the entire server remotely), I landed on DirectAdmin reading at a lot of places, mostly because I found Cpanel rates had become very abusive (but I was wrong you pay for what you get, now I know CPanel is ridiculously stable and that is why it is so expensive).

With DirectAdmin from the beginning a task as simple as configuring domains failed, people at Tech support had to make them work from their end , never telling me what was the issue, domains would be supposedly created but they would never go show on the list or go live and when trying to create them again (with them not showing anywhere) I would get a message saying they could not be created because they already exist.
Trying to install modules via custombuild fails a lot and you have to end up tweaking manually, thing I never ever did in 20 years using Cpanel.

As far ad the Fail2Ban vs CSF issue, I will look more into it I am going by what I read at many places today for protection when your SSH is getting attacked and you don't want to or can go with a hardware firewall option. Apparently Fail2Ban gathers information from all log files and creates ban lists with different levels of customization. I have not looked at CSF in detail to see if it can do that or not, I do not dislike CSF, my problem is CSF I had configured from that module in DirectAdmin and as I said I have no access to it, I can always edit it via Linux, but so far in all my research no one suggested to use CSF for massive SSH attacks and the ban list, everywhere I read I found Fail2Ban as a suggestion (combined with other things I am implementing}. I will try the IP tables suggestion and come back with my findings.

But the bigger question is, why did that happen int he first place? How did my server go from me being able to do SSH connections and Directadmin management and magically that disappeared. That is the sort of things that keeps happening with DirectAdmin. Everything is ok and then magically it is not. Or configuring the simplest of things that in CPanel happen basically with 2 clicks require massive research and a lot of programming to get to work in DirectAdmin.

And when going to read a the forum I find very often the half answers of a smartass that I don't know if he works for DirectAdmin or what the deal is, many of the guides for specific tasks are incomplete and when using the exact instructions they do not work (for stuff that with Cpanel happens automatically).

It isn't a matter of having or not having the knowledge to do something, it is about why you have to do it in the first place, this product competes against one that does not require you to go though so many hassles for most things.

And to the guy who mentioned I said "nothing works anymore" I never said that, so far I have a super sophisticated/complex website working perfectly and with a massive amout of modules and functions, but every single step to build that was unnecessarily complicated with DirectAdmin and randomly I end up having major issues (without touching a thing), in this case magically I am locked out of being able to manage the Web Server Panel, and again I have to dedicate significant time finding out now what the f@#$k happened with this thing...

27 years in this business and this is only the second system (after cyberpanel which was far worse BTW) where I see how things magically explode for no reason whatsoever and every ttime that happens I find myself fixing what should not be broken.

This with a fresh linux install with nothing else in it but directadmin, in my very own massive web server at a state of the art facility.

 

[JGSFla]

Fail2Ban and CSF do the same thing, and will likely conflict and be confusing. I'd recommend sticking to one or the other.

I'd probably eliminate the entire firewall first. Normally I'd suggest disabling CSF but you have Fail2Ban running too, so might be easier to just add a rule that bypasses everything else.

iptables -I INPUT -p tcp --dport 2324 -j ACCEPT

I can't tell from your post if you have SSH working now or not, but you could repeat the command for the SSH port.

Thanks, the useful reply here.

 

[JGSFla]

Did you try to check if the ports are open with this tool

Offtopic,

Honestly, I don't understand your story. you indicate that you made a mistake moving to directadmin and that inevitably nothing works anymore. It also seems to me that you don't understand anything about it and barely have any knowledge. maybe someone on the forum can help you with paid emergency support.

I deployed my first Apache server from the ground up in 1997, hosting 5088 websites, when you used to do zone files, conf file and all of it manually, also an IIS server deployment specialist at the time. Have developed over one housand complex wesbites in 14 countries. More extensive experience probably than any of you here in many panels, I started using Plesk, then Cpanel to make my life easier, not due to a lack of knowledge. Have used extensively Plesk, Cpanel, Webuzzo, Recently Cyberpanel and now DirectAdmin.
And yes I can do whatever needed in Linux, my web servers I assemble from the ground up and tweak all the software I install, also have a group of people to do remote hands support on my servers at the co-loicationf acilities I use.

"Paid emergency support", first it is not an emergency because my site is working properly, second I do have access to my server via my remote desktop if I need to and I do know linux so I can take care of whatever I need, the point is why does this thing fail out of the blue? why is it so unstable? why the direct tech support gives half answers always. Sounds like a poorly developed product purposely to sell support.

The huge number of issues I have had with DirectAdmin are far far from the norm, it is a unstable system and it is very incomplete as far as guides to do specific tasks, you have to find yourself doing a lot of research to get very basic things to work (e.g Redis cache). Creating the first domains when I deployed the server failed (yes the most basic of functions) requiring people at tech support to fix them, never telling me why that failed.

 

[JGSFla]

Most likely because you really have to be a real admin which most cpanel admins are not, as they almost never have to use SSH and do admin things which are not done via the GUI. Because cPanel does everything for you and otherwise cPanel crew can be asked to look.
Because most of us don't have any issues with Directadmin. However that doesn't help you now.

But be aware that if you drive a Ferrari and you want to go drive an Mercedes, it's quite different under the hood and you don't pay Ferrari service and motor. Still Mercedes is a very great car.

First of all, the most important thing you forgot to mention is -why- you can't access Directadmin via SSH or GUI as admin, otherwise said, you didn't post any error notice given. We can't guess what's going wrong then.

Maybe the problems were already caused by the attack, before the attack was really noticable by you. All kinds of options are possible, no need to directly blaim DA.

Anyway, if you can do linux things in another way that is a good start. First of all I would check these:
- In the sshd_config is the port still 2425, in case of emergency, have 22 also open in CSF (both incoming and outgoing)
- Doublecheck this port is still enabled in CSF, test anyway with the default port 22 in case of datacenter blocking of other port for whatever reason
- Check if PermitRootLogin is set to yes (at least so you can login some way, even if it's temporarily)
- Check with iptables -L if all is correct.
- Ofcourse check the logfiles for any error messages when trying to login as admin in DA and as root do SSH. I presume you know what logs to look at.

And if you reply back, then don't forget to mention errors seen or notices when trying to login via SSH and Directadmin GUI as admin.

You already had permission issues in the beginning (click) which better had been fixed then. Might have been caused by installing directadmin and not waiting long enough for the background installation to finish, so I hope you waited long enough. But already with those issues you should have send in a ticket.

You might indeed want to consider to hire some professional help. Also guys from the forum like @zEitEr's and @smtalk's company's can be hired. And Rack911 is also on the partner page.

If you really think DA is to blaim, I would also send in a ticket to DA's ticket system.

27 years of experience here, programmer knowing 4 computer languages, have deployed tens of large enterprise servers and web servers, used to be IT manager for several major companies. And over 20 years of experience with Plesk and Cpanel, have also deployed servers using Webuzzo, Cyberpanel and now DirectAdmin.
DirectAdmin is unstable and makes simple tasks very complicated, guides to configure modules are very incomplete.

 

[JGSFla]

Thanks, the useful reply here.

Fail2Ban and CSF do the same thing, and will likely conflict and be confusing. I'd recommend sticking to one or the other.

I'd probably eliminate the entire firewall first. Normally I'd suggest disabling CSF but you have Fail2Ban running too, so might be easier to just add a rule that bypasses everything else.

iptables -I INPUT -p tcp --dport 2324 -j ACCEPT

I can't tell from your post if you have SSH working now or not, but you could repeat the command for the SSH port.

As I said I did have remote desktop access via iLO4 (iLO5 and iLO5) are the management consoles for Hewlett Packard servers for those of you here who don't know (so no need for SSH because iLO allows me operate the server remotely).

I opened it via IPTABLES as you suggested and it re-opened the port so I regained access to DirectAdmin, thanks so much for that.

The question is why did that one and the other one were blocked without touching anything (the instability I refer to of DirectAdmin). I checked log files for successful and failed attempts to login in those ports and I had been the only one successful in login in vie SSH or to the DirectAdmin console in the past month (after thousands of other attacks), those ports were not even targeted, so why did that lose the configuration for no reason? I had not loggged in to directadmin or used ssh access in weeks and suddenly it was locked out.

As far as Fail2Ban and CSF doing the same thing and incompatibility, I did extensive research and I can actually configure CSF to do the same thing Fail2Ban was going to do (create a blocked IPs list for brute force attacks) so I will configure that portion of CSF instead and remove Fail2Ban.

I think the material I found was dated and in the past Fail2Ban used to be the standard for those brute force attacks but now CSF + LFD has a portion of it where you can configure that (although I see that was not the reason for the problem, those ports weren't even attacked).

Thanks for the help.

I wrote many other replies to people here but this system's moderator holds them or blocks them.

 

[adam12]

It's hard to say what happened since we don't know which firewall added your IP. If you're able to figure that out, then it might be possible to narrow it down some.

I would definitely uninstall Fail2Ban. CSF is essentially Fail2Ban plus a bunch of other great stuff. One CSF feature you might look at enabling is having it add firewall rules by port (instead of a blanket blocking). That way, you might be able to get back into SSH if it trips again (or vice-versa).

If you have a semi-static IP, I would see if you can add it to the allowlist in CSF to prevent you from being blocked again as well.

 

[Richard G]

I opened it via IPTABLES as you suggested and it re-opened the port so I regained access to DirectAdmin, thanks so much for that.

So have you checked the csf.conf file if the port was still present in there?
Because if you use the line suggested and it works, then this means that CSF did -not- work as should be or the port was not in the config (or not anymore).

You can keep referring to Directadmin's instability but it's hardly believable that DA would change anything in this. Next to the already known root issues you had before.

I don't know if you have other admins with access to the firewall via the GUI or why it could have changed because DA does not touch the firewall itself. Or something like immunify360 which might have messed things up?

It's indeed good to only use one firewall and csf can do everything fail2ban can do and more, so best is to stick with CSF.

However reading the other issues before where root even had no permission in the other thread, I guess you might encounter other odd issues. To me it sounds something is wrong with that installation and might be a good idea to have it checked by DA.

If you have a semi-static IP, I would see if you can add it to the allowlist in CSF to prevent you from being blocked again as well.

And the csf.ignore list too. Unless they are combined in csf.conf then the csf.allow would be enough.

 

[JGSFla]

So have you checked the csf.conf file if the port was still present in there?
Because if you use the line suggested and it works, then this means that CSF did -not- work as should be or the port was not in the config (or not anymore).

You can keep referring to Directadmin's instability but it's hardly believable that DA would change anything in this. Next to the already known root issues you had before.

I don't know if you have other admins with access to the firewall via the GUI or why it could have changed because DA does not touch the firewall itself. Or something like immunify360 which might have messed things up?

It's indeed good to only use one firewall and csf can do everything fail2ban can do and more, so best is to stick with CSF.

However reading the other issues before where root even had no permission in the other thread, I guess you might encounter other odd issues. To me it sounds something is wrong with that installation and might be a good idea to have it checked by DA.


And the csf.ignore list too. Unless they are combined in csf.conf then the csf.allow would be enough.

I don't allow anybody other than myself to touch that specific server, it is too important. In case of emergencies I give temporary access then revoke it when people are done.

As far as the firewall, until this morning when I noticed the problem it was only CSF I added Fail2Ban this morning because I logged in to the server via remote access and I was getting a crazy amount of attacks showing real time on the terminal ( I had to move to F2, no patience for that and stop the verbose response to that).

The reason I blame DirectAdmin is because I have been doing this for so long, in tens of servers and thousands of sites and I never had such problems with the 2 major panels out there. Almost identical setups for most of the stuff. Maybe DirectAdmin does not play nice with AlmaLinux. It failed from the first hour of using it and it has overcomplicated things every step of the way, and it randomly has major issues like this one just out of the blue, not touching anything.

 

[JGSFla]

I don't allow anybody other than myself to touch that specific server, it is too important. In case of emergencies I give temporary access then revoke it when people are done.

As far as the firewall, until this morning when I noticed the problem it was only CSF I added Fail2Ban this morning because I logged in to the server via remote access and I was getting a crazy amount of attacks showing real time on the terminal ( I had to move to F2, no patience for that and stop the verbose response to that).

The reason I blame DirectAdmin is because I have been doing this for so long, in tens of servers and thousands of sites and I never had such problems with the 2 major panels out there. Almost identical setups for most of the stuff. Maybe DirectAdmin does not play nice with AlmaLinux. It failed from the first hour of using it and it has overcomplicated things every step of the way, and it randomly has major issues like this one just out of the blue, not touching anything.

And one more thing, the CSF rules worked fine for I would say 6 months, I did plenty of updates in DirectAdmin, some regular maintenance on it andI logged in many times via SSH to do simple things in my server, I rebooted that server a bunch of times and I restarted services many times, etc.. I have not checked the csf.conf file , will do tomorrow, but whatever happened happened for no reason whatsoever, it just lost that configuration apparently. And the CSF module is running from DirectAdmin, that is why the blame.

 

[johannes]

If its not CSF/LFD, whats about BFM? https://docs.directadmin.com/directadmin/general-usage/securing-with-bfm.html
Check your IP in /usr/local/directadmin/data/admin/ip_blacklist
(or in the BFM-section in the GUI, if the file is missing)

 

[exlhost]

DirectAdmin is unstable and makes simple tasks very complicated, guides to configure modules are very incomplete.

In the 15 years that we have been using DirectAdmin, I have never had any problems. We run more than 10 servers with AlmaLinux and DirectAdmin without any problems.

 

[JGSFla]

In the 15 years that we have been using DirectAdmin, I have never had any problems. We run more than 10 servers with AlmaLinux and DirectAdmin without any problems.

Could sound like I am one of the "unlucky" ones until you start browsing online and find hundreds of people with similar problems...(every time I have any kind of problem).

I don't know what the scope of your activities is with those servers but some of the things I run are complex and I never had such problems with CPanel or Plesk. I added the modules I needed, did some tweaking and it just worked, no Chaos, no mysterious problems not allowing things to run, no sudden changes locking me out.

My only complain was pricing and more recently the "Solo" license for some of the Cpanel servers I had because that structure makes for a very dirty way to create domains (where one is a main domain and makes the rest child domains and sort of sub-domains making for super dirty zone files, certificates, etc..).

What I "saved" in this DirectAdmin experiment more than had to pay in remote hands emergency troubleshooting without accounting for my own time which is worth 20 times that of what I pay for remote hands. If it wasn't such a huge task to move back to CPanel I would have done it the first week after installing DirectAmin.

What's done is done when I have the time and patience I will migrate back to CPanel and in the mean time I'll suffer through all these absurd hassles.

Plus I don't know if you actually have servers or if you have VPS accounts, I do have both, actual servers (the machines in racks) in data centers and some VPS accounts with some companies. This DirectAdmin issue is specifically in a very large dedicated server I have at a Tier 3 facility, and my server has like 100 times more capacity than what I need (in terms of specs, cores. threads, memory, storage) and it is constantly monitored for performance, I am not using even 2% of its total resources ever, the only thing that is constantly failing is DirectAdmin.

 

[JGSFla]

If its not CSF/LFD, whats about BFM? https://docs.directadmin.com/directadmin/general-usage/securing-with-bfm.html
Check your IP in /usr/local/directadmin/data/admin/ip_blacklist
(or in the BFM-section in the GUI, if the file is missing)

Can't be IP related because I tried first from South Florida then I had people in 2 data centers one in Chicago and one in San Francisco test for me and they had problems as well.
And more importantly, my IP is static and only 5 people use it and only I access that server, and I had not accessed it in a couple of weeks, then I was locked out.

And of course we thought it could be related to CSF defaulting to original settings or whatever and tried to use port 22 and port 2222 respectively but the only difference we found is in the default ports the bounce would be instantaneous and with the custom ports it would take some time and the attempts would time out eventually.

But as I mentioned before (not sure if that post was "authorized" by the moderator" here) the issue is it isn't a matter of whether we have the knowledge or experience to eventually find what is the problem and fix it, it is the fact that these absurd problems keep popping up with this panel. We don't have time for that. It does not happen with Plesk and Cpanel, not ever in more than 20 years using them. This thing failed from the first hour of using it, wouldn't even create domains.

 

[Richard G]

DirectAdmin does not play nice with AlmaLinux. It failed from the first hour of using

DA plays great with Alma 8 and 9, most of us here are running it. But you say it failed from the first hour of using, which makes me wonder stronger if you waited using DA or waited configuring it, until all background tasks were finished. Because nowadays DA is installed very quickly, but all the rest is done in the background and takes time.

And the problem is that if you already start configuring things then, it most likely will lead to issues as for example apache or mariadb is not ready then yet and certain info is not passed to them. It was better before when it was not done in the background, so people were forced to wait.
Or something else went wrong, however there are thousends out there using DA. If DA was crap, then also the forum would be overwhelmed with all kind of weird issues, which in fact it's not.

Sudden locking out, other then firewall or DA bruteforce blocking (but then you're totally blocked in both cases) has never happened to my memory of what I've seen on the forums. You still could visit your Wordpress, so it was no full block.
And DA never ever removed any line from a firewall.

Which is why it's so important to see the notices you got at the time you couldn't log in and what the logfiles said at that particular moment.
Because it can't be the DA bruteforce blacklist either, otherwise only an iptables command wouldn't have let you get into the GUI again.
I presume you check the /usr/local/directadmin/data/admin/ip_blacklist file already, but that should not contain your ip.

Also in the meantime I do believe you have the knowledge, it just I've seen this too often lately from cPanel users coming over. Which is why I started asking if you waited configuring DA until all background tasks were finished.
And that combined with your other post where already very odd things were happening leaded me to my thought that you did not wait long enough, which happens to a lot of people, even experience once. Also 1 time to me after this change of installation.

And also this strenghts me in my thought that you didn't waited until installation was finished in the background:

This thing failed from the first hour of using it, wouldn't even create domains.

Because that is -exactly- what can give issues if you start right after DA itself is finished and the rest isn't yet. Or otherwise at least a good reason to doublecheck things and ring the bell on ticket support.

You wanted to know if I had vps or servers. We do have servers, at the moment we are running 5 DA dedi servers, I'm managing another 2 DA servers from another company and managing a cPanel server from again another company and managing 7 non-hosting servers. And we have only 1 vps for nameserver purposes.
I personally started with DA in 2008 which is 16 years now and never had such odd issues like root with no permission or dissapearing firewall rules. Neither remember seeing something like this ever on the forums. So I hope it's not disk or ram issues causing this. Because then you will have a ball with another panel too.

Edit: added 2 lines and fixed a typo.

 

[JGSFla]

DA plays great with Alma 8 and 9, most of us here are running it. But you say it failed from the first hour of using, which makes me wonder stronger if you waited using DA or waited configuring it, until all background tasks were finished. Because nowadays DA is installed very quickly, but all the rest is done in the background and takes time.

And the problem is that if you already start configuring things then, it most likely will lead to issues as for example apache or mariadb is not ready then yet and certain info is not passed to them. It was better before when it was not done in the background, so people were forced to wait.
Or something else went wrong, however there are thousends out there using DA. If DA was crap, then also the forum would be overwhelmed with all kind of weird issues, which in fact it's not.

Sudden locking out, other then firewall or DA bruteforce blocking (but then you're totally blocked in both cases) has never happened to my memory of what I've seen on the forums. You still could visit your Wordpress, so it was no full block.
And DA never ever removed any line from a firewall.

Which is why it's so important to see the notices you got at the time you couldn't log in and what the logfiles said at that particular moment.
Because it can't be the DA bruteforce blacklist either, otherwise only an iptables command wouldn't have let you get into the GUI again.
I presume you check the /usr/local/directadmin/data/admin/ip_blacklist file already, but that should not contain your ip.

Also in the meantime I do believe you have the knowledge, it just I've seen this too often lately from cPanel users coming over. Which is why I started asking if you waited configuring DA until all background tasks were finished.
And that combined with your other post where already very odd things were happening leaded me to my thought that you did not wait long enough, which happens to a lot of people, even experience once. Also 1 time to me after this change of installation.

And also this strenghts me in my thought that you didn't waited until installation was finished in the background:

Because that is -exactly- what can give issues if you start right after DA itself is finished and the rest isn't yet. Or otherwise at least a good reason to doublecheck things and ring the bell on ticket support.

You wanted to know if I had vps or servers. We do have servers, at the moment we are running 5 DA dedi servers, I'm managing another 2 DA servers from another company and managing a cPanel server from again another company and managing 7 non-hosting servers. And we have only 1 vps for nameserver purposes.
I personally started with DA in 2008 which is 16 years now and never had such odd issues like root with no permission or dissapearing firewall rules. Neither remember seeing something like this ever on the forums. So I hope it's not disk or ram issues causing this. Because then you will have a ball with another panel too.

Edit: added 2 lines and fixed a typo.

What you are saying makes sense as I am not known to have patience (172 IQ), I am known to do stuff too fast and yes I had no clue this thing would install anything in the background, especially having a server way above my needs for this specific purpose. To me if something say it is complete oh well, it is complete, and I move on and try to get done with everything as fast as possible.

Also, this specific application was meant for speed which means I was not going to use Apache but NGINX or Litespeed and Memcached or Redis. Also needed very specific modules to speed up image conversion, compression, etc..

It is sounding more and more (because I read far more stuff last night not only this thread) what I am doing is sort of unconventional for DirectAdmin and too much of it (that is why I find the install guides and instructions so poor because I am doing unconventional things), if you add to it the fact that perhaps it did not install completely (because I think my hardware is pretty vanilla so that would not be the source of problems) it is a recipe for disaster. It is now sounding like I have a corrupted install from the beginning because I did lots of things too fast and the thing was not completely deployed (based on what you are saying about background stuff still installing).

Add to that perhaps because of the fact that the install was corrupted many of the modules I tried to install via custombuild would not work properly and I ended up doing them manually via Linux myself, yes could be that install is now all messed up , hence super unstable. I can see a scenario where a few things are double installed , perhaps older versions with newer or versions not meant for DirecAdmin, and I was not about to verify every single thing that did not install properly with custom build manually, I would simply start and enable services and see the response messages (hence why I never got a few things to properly work, they work in Linux but not in DirectAdmin, there is no link to them internally in the web server).

I remember several instances where custombuild finished installing I did my service restarts etc.. and the stuff simply would not work so I would have to end up installing it manually in Linux and then having to do configuration files manually to try to link them to DirectAdmin and still have them not being seen by DirectAdmin, reading tens of forums where the explanations I was finding were conflicting with each other and me testing each one of those theories. Some times, in the end, not being able to get modules to work at all. And furious because those same modules in my CPanel systems take literally 2 clicks to install and run.

Plus being an user of many many platforms in 30 years I quickly see design flaws (like how complicated it is to change the security of the page where you log in or the time you stay logged in, or how you log out), or how the domains are displayed or controlled (super archaic and basic compared to other systems). Add to that the fact that I had just completely removed Cyberpanel from that server (my previous experiment) and it was a complete debacle so my patience was pretty low.

But the most important thing that has made me have the wrong impression about the system, I am a super sophisticated user due to my background and what I am trying to accomplish, I went for the $5 a month license because it was never my intention to request support, then the domains never got properly created and I had to request support, DA was saying I could not create domains because they already exist yet they where showing nowhere. The tech support guy fixed them in like 10 minutes but when I asked why it happened and where was the problem I got no answer ( I need answers to move on you know, and prevent future incidents). A few days later I could simply not get Redis to work, read all manuals all forums, it was showing as working in linux but the plugin in Wordpress just would not see it, and I talked to tech support and by then they told me if I wanted support I needed to pay them and I was like what??!! your stuff is not working following the exact directions in your materials, it was then like the third time that happened. So I have to pay support because the product does not work properly? Sounds like a scam to me.

When I have more spare time I will be looking at some log files. The initial list I think I should check is this (found in a very interesting page):
SSHD_LOG = "/var/log/auth.log"
SU_LOG = "/var/log/auth.log"
FTPD_LOG = "/var/log/syslog"
SMTPAUTH_LOG = "/var/log/secure"
POP3D_LOG = "/var/log/mail.log"
IMAPD_LOG = "/var/log/mail.log"
IPTABLES_LOG = "/var/log/syslog"
SUHOSIN_LOG = "/var/log/syslog"
BIND_LOG = "/var/log/syslog"
SYSLOG_LOG = "/var/log/syslog"
WEBMIN_LOG = "/var/log/auth.log"

That should start getting me a better idea of how the server is being attacked and develop some theories. But again, that is not my job...

And if you are correct and the background install is the problem then that is a major flaw in that system, because for lots of people like myself if you say something is complete, well, it is complete. If you are installing (especially on a new server deploy) 30 things you are not supposed to babysit all of them, not your job. The software should be the one doing a clean up job at the end , making some sort of inventory, a final report and then if it is truly complete it returns a message saying it is complete and if it is not it tells you what went wrong so you don't use it until everything is right.

To me, this system (DA) looks half cooked, and the explanation of the background stuff not properly finishing installing is just one more example of it.

 

[Richard G]

What you are saying makes sense as I am not known to have patience (172 IQ), I am known to do stuff too fast and yes I had no clue this thing would

Well... that's a bit of a problem, because this is something which happens both with noobs and with professionals (as they know how things work and often are less patient or read less manuals). Like I said, also happened to me the first time. So I guess we can carefully say that is the root cause of the isseus encountered.
However most cPanel admins we see here... lets say the ones with a enough knowledge coming ove are scarce here. You're one of the few exceptions to this.
Indeed it's expectable that you're in a bit more haste and already pre-irritated coming from another debacle. Who wouldn't be.

It is now sounding like I have a corrupted install from the beginning because I did lots of things too fast and the thing was not completely deployed (based on what you are saying about background stuff still installing).

Unfortunately that seems indeed the case. And things you installed manually then most likely only added to the problem. And makes it that at this moment there is no easy fix. Otherwise we might have be able to try a forced "build all d" and then resetting permissions and rewrite_confs and probably things would have been fine then. But I doubt that will work now.
And since there are manually additions/installations done, I won't try to advise it because it might mess things up more and I don't want you to get more trouble as you already have.

The initial list I think I should check is this

Only syslog and ssh log maybe. Because those are the logs you find in the csf.conf file. The list too look at is indeed the syslog (if you're on debian) but also logs present in the /var/log/directadmin directory, which also state what happens on login attempts and issues. Various logs in there.
Probably they will contain loads of errors because the initial install already wasn't good enough.

if you say something is complete, well, it is complete.

I most certainly agree with you here! It's said that you can start configuring then already. But it's prooven that this won't work. Yes you can set some basic settings for DA itself already but not really start like one would expect.
It would at least be good to issue a very clear warning to limit all actions to only DA related settings until installation is totally complete.

For the rest... well... DA is cheap and works great but needs some more attention. But well... as with all things. Price differences have reasons.

I'm sure if you would start over with DA, now knowing you have to wait, and it's not everything click and play like cPanel, you would appreciate what you get for the price, as do thousends of others, also big firms and people with kindlike knowledge.
It's a choice.