CustomBuild and DirectAdmin integration discussion

[sparek]

I agree with @fln that the thread from DirectAdmin v1.646 has been released should probably stay on the topic of issues with DirectAdmin 1.646.

I suppose it was me who diverted that thread into a different discussion and I apologize for that. Originally I had thought that a bug existed somewhere that updated my servers to 1.646 without my explicit consent. But this turned out to be a /usr/local/directadmin/custombuild/build update in a daily update checker that did the upgrade. That itself is also kind of its own issue.

The theme that I see regarding this discussion has to do with foresight or the lack thereof. Or perhaps it's just unreasonable expectations by the users of the software.

Let's start out by defining what a web hosting control panel is. Perhaps everybody's definition is different from my own and maybe that's part of the problem. For DirectAdmin, I consider the "control panel" to be everything that is accessed on port 2222 (although this is customizable, it is the default DirectAdmin port). For cPanel, the "control panel" is everything on ports 2083, 2087, and 2096. Does that make sense to everybody?

DirectAdmin, cPanel, and every other control panel does not develop Apache. They do not develop nginx. They do not develop Litespeed. They do not develop MariaDB. They do not develop MySQL. They do not develop Roundcube. They do not develop Dovecot. They do not develop Exim. But they do develop the stuff that is accessed via the respective port numbers in the last paragraph. That's why I consider that stuff as being the "control panel" for that respective control panel solution.

The control panel may interface with those applications and as such it's OK for a control panel to say they only support these web servers, these sql services, this IMAP service, and this MTA. As far as I know neither DirectAdmin nor cPanel support Lighttpd. If I want to use Lighttpd on my server, then that's on me to figure out how to do it. And through DirectAdmin and cPanel's hookable events - I ultimately think this could be possible. Other control panels that do not have such widespread hookable events maybe not. The point here is that essentially everything these control panels do to interface with these stack applications is doable through a hookable event. But instead of forcing users to do this on their own or doing this through an editable and viewable open source context - they just do it within their own control panel code.

One of the key differences I see with DirectAdmin is that the DirectAdmin control panel (i.e. port 2222) is not self-contained. Meaning it depends on so much from the system - and likely the applications from the stack in CustomBuild - rather than ship out its own systems. DirectAdmin always seems to get in a fuss when it's compared to cPanel - but as an example, cPanel is self-contained. They use their own PHP for stuff run inside their control panel (ports 2083, 2087, and 2096) - /usr/local/cpanel/3rdparty/bin/php. And outside of interfacing with the MySQL/MariaDB service for end-users, they pretty much store everything in either SQLite database - which is also controlled by their own binary - /usr/local/cpanel/3rdparty/bin/sqlite3 - or they use extensive YAML or JSON files. The point is the cPanel control panel services (ports 2083, 2087, and 2096) don't really care what web server you are using for your customers, what PHP version you are using, what sql service you are using - it's going to continue to operate regardless of that. Again... if I choose to use Lighttpd as my web server, cPanel's not going to auto create those VirtualHosts - but the cPanel services themselves (ports 2083, 2087, and 2096) are going to continue to operate themselves.

So, I suppose my question in this quandary, why does the DirectAdmin control panel have to rely so much on the applications in the server stack? Or perhaps it's just unrealistic expectations for me to expect DirectAdmin to behave this way.

Looking at the software provided by CustomBuild, and maybe this isn't a full list:

Jailshell
Apache
Pure-FTPD
ModSecurity
dovecot
dovecot.conf
Exim
exim.conf
SpamAssassin
MariaDB
PHP
RoundCube
phpMyAdmin
ionCube

Then I start to pick some of these out:

Jailshell - could this be dependent on the DirectAdmin version? Quite possibly. Why is it an option in CustomBuild then? Remove it from CustomBuild and it gets updated when DirectAdmin gets updated.

phpMyAdmin - again, this should probably be more contained within the control panel - but it's not. Why is it in CustomBuild? Remove it and integrate it into DirectAdmin and update it with DirectAdmin. If an account wants to access phpMyAdmin not from the control panel, then they can install phpMyAdmin on their own within their domain's DocumentRoot.

Roundcube - This one is interesting. DirectAdmin doesn't have a facility for webmail like cPanel does (i.e. port 2096), so the argument can definitely be made that Roundcube is outside the jurisdiction of the DirectAdmin control panel. BUT... This Roundcube depends on a server-wide SQL database - da_roundcube - which is integral to the DirectAdmin control panel. This would definitely be an interesting topic for discussion on how to better handle this. Should DirectAdmin do something like cPanel does with port 2096 and self-contain Roundcube into the DirectAdmin control panel and run Roundcube from it's own web-server on a specific port, then allowing Roundcube updates within the DirectAdmin updates? Or is there another solution?

Everything else listed in CustomBuild - I question, why does the DirectAdmin control panel need to be involved with this?

Even something as simple as MariaDB 10.3 support in CB and old DA brought major problems due to structural changes (mysql.users became a view instead of real table), same happened with MySQL8 and there are many other examples.

Admittedly, I'm at a disadvantage here - I'm still running MariaDB 10.3 on all of our servers and mysql.user is still a table in this version. But looking at our cPanel servers show MariaDB upgrade options of 10.5 and 10.6 as well as the current 10.3. So obviously they are able to deal with mysql.user as a table and a view. But also to your point - these major updates (i.e. 10.3 to 10.5 or 10.3 to 10.6) are controlled within the control panel itself. So perhaps that is an option for DirectAdmin to take with this. Allow control of the MariaDB minor versions from within the DirectAdmin control panel (i.e 10.3, 10.4, 10.5, 10.6, etc) but allow patch versions (i.e. 10.3.37, 10.3.38, 10.3.39) to be updateable within CustomBuild. If you later release a DirectAdmin 1.5 that kills support for MariaDB 10.3 (which goes end-of-life this May) then that entails a blocker preventing an update to DirectAdmin 1.5 if the system is still on MariaDB 10.3. You could do this with the other sofwares in the stack if so desired.

Does not it mean it requires an update of the panel? With the "biggest" I guess you've meant https://docs.cpanel.net/release-notes/104-release-notes/, here I also see entries like:

Roundcube upgrade to 1.5
In cPanel & WHM version 104, we upgraded Roundcube to version 1.5.2. This release includes a spam marking/unmarking feature.

Does not it mean cPanel&WHM 104 is required to run RoundCube 1.5? It makes sense to me, as they can be guaranteed that control panel fully supports it. It'd be appreciated if you may explain how it works then and what did you mean with "separate panel and software update". Thank you.

This actually does make sense to me - because cPanel controls its webmail (port 2096) within itself. I'll admit the DirectAdmin Roundcube/Webmail issue is both an interesting dilemma and paints the picture of DirectAdmin not being self-contained and the problems there in.

Ultimately, I think a lot of these self-containing issues draw from the fact that DirectAdmin is installable on so many different OSes. So how do you write a single DirectAdmin project that aims to run on so many different OS variants? This speaks to the foresight theme I mentioned earlier. Not understanding the problems that would arise while trying to support a product across these different OSes, likely lead to or at least aided the inability to design DirectAdmin as a self-contained product. You've dropped FreeBSD support - which is probably a good thing. But you still have RedHat-variant support and Debian-variant support. That's going to involve a lot more development in building the product to be self-contained within two different environments.

I don't know what the licensing numbers look like. I don't know how many Debian-variant licenses you have versus Redhat-variant licenses. Or another way to look at it might be the total number of accounts on Redhat-variant licensed servers versus the total number of accounts on Debian-variant licensed servers. I would suspect, and maybe I'm wrong, that the Redhat-variant (RHEL, CentOS, AlmaLinux, CloudLinux, Rocky Linux) would far outnumber in terms of total accounts on those servers when compared with Debian-variant (Debian, Ubuntu) licensed servers. I think the majority of production-level web hosting company servers are going to be Redhat-variant based. And while I'm sure there are web hosting companies that use Debian-variants with a high number of accounts, I would suspect that a lot of the Debian-variant licenses would be smaller, hobbyist-like servers with few accounts.

Perhaps a solution is to fork DirectAdmin into another "more web hosting company production" centric. Make it solely Redhat-variant based. Build it so the control panel is self-contained. Build it so CustomBuild (I'd love to see an RPM package based release system) can be separate from the control panel.

Or perhaps what DirectAdmin is just doesn't fall into the expectations that "web hosting companies" expect. And expecting DirectAdmin to be a solution is a wrong presumption.

 

[Ohm J]

Everything else listed in CustomBuild - I question, why does the DirectAdmin control panel need to be involved with this?

Because custombuild create for building and compiler the package from source code.

Control Panel is just UI Application. That's why custombuild 2.0 merge into Directadmin banary. because it GUI interface.

If you like RPM Base Package, then how do you install PHP7 on new OS like Almalinux9 ? ( There have some guy want to run old PHP version even EOL. ).
How do you add custom Nginx Module and build it ? No you can't.

that's why building from source code better than RPM package.

 

[Richard G]

Control Panel is just UI Application. That's why custombuild 2.0 merge into Directadmin banary. because it GUI interface.

I think we have to split 2 things which also made things confusing in the past.
There is the custombuild as it always was, in which you could update and config all kinds of things like mentioned in the list.

And then there is the custombuild plugin, imho more invented to make live easier for less technical admins.

It seems the control panel is not just the UI application, as they want to integrate the complete Custombuild into Directadmin as far as I understood it, not only the plugin.

If you like RPM Base Package, then how do you install PHP7 on new OS like Almalinux9 ?

Well... this is also not possible with DA, because Alma 9 uses Openssl 3.0 which is not compatible with anything lower than PHP 8.1.

However RPM updates is the big difference with cPanel. They make their own RPM's which is why using RPM's is possible at CP.
There is nothing wrong with source compiling some things as Directadmin does and this way supporting some older stuff.
It's not better always, often the OS updates (so RPM) is better.

Not everything is build from source. For example mysql and mariaDB is build from RPM. So there are positive things to say for both, depending on what you're doing with them.

I'm an old guy and maybe got too used to the old nice system of DA where you could do things yourself, which also created some customisability and it's light, not using much resources.
Also another benefit is that you can see what you're doing and can see when it goes wrong and what goes wrong.

However, I only don't see any future in combining the panel binary and the updates fully. In my opinion some way must be kept to be able to skip newer DA versions if needed, and update important apache/php/mysql/whatever updates.
It would be nice if they build all together, that this might be done for example via a commandline option or some feature where you can make a choice to do so in the new system. Without going back to the 2 seperate systems as it was before, since the choice is made to build all together.

 

[Ohm J]

I agree with you. It should update important service ( php, sql, ..etc.. ). without updating Directadmin binary.

So DA team should make option to choose between get custombuild version from DA binary or get from the link ( files.directadmin.com ) like in the past.

with this, We can testing some important service before push update to DA binary Release version to the people that use custombuild from DA binary.

 

[Richard G]

like in the past.

They don't want to go back to the past.
But if they can do it by adjusting the ./build update command, so ./build update and ./build update_versions would only update apps and ./build update_da (or something similar) would update the DA binary, that would also be a solution.

However, I don't know if something like this is possible in the system they want to create.

 

[sparek]

I agree with you. It should update important service ( php, sql, ..etc.. ). without updating Directadmin binary.

That's kind of the point of this discussion.

I listed the applications in CustomBuild - maybe it's not a complete list, but it serves the purpose of offering an example of the discussion.

To draw from that and another example: Apache is currently at version 2.4.54. If Apache releases a new version tomorrow - 2.4.55 - why do I have to update DirectAdmin to get that update? What if I want to wait for all of the headaches that a new DirectAdmin control panel update brings to pass before updating? This would have been doable previously when CustomBuild and DirectAdmin were separate.

The only explanation that I've seen is a vague "CustomBuild needed to be integrated into DirectAdmin, so we could offer new features in DirectAdmin." Why? Why can't the DirectAdmin control panel be it's own system, self-contained? What is the DirectAdmin control panel drawing from the CustomBuild applications that is required for it to be working? Why not simply install or package those necessary applications somewhere else, with each DirectAdmin update, so that the entire DirectAdmin control panel is self-contained?

 

[sparek]

Our goal is to make smaller releases more often, but right now we are making bigger structural changes to the code-base so it takes some time for new features to accumulate .

This little nugget may also play a role in this.

I don't know if I can speak for all of the web hosting companies out there using DirectAdmin. But for me... I prefer stable constants and I think our clients do too. Changing things too often tends to bring back customer backlash for us.

Perhaps this speaks to the market that DirectAdmin is aiming at - and thus the unrealistic expectations of like-minded web hosting companies using DirectAdmin. Servers that have 10 accounts on them or servers where the "admin" is also the "webmaster" for all of the accounts on the server they are better equipped to handle more frequent changes. Servers that have 1000+ accounts and where the admin-to-user relationship is defined more as "paying customer" it's harder to accept change. I use the allusion of turning a boat at sea. A jet ski or pontoon boat is going to be able to turn more on a dime than a cargo ship. The jet ski or boat size alludes to the number of accounts on the server and the turn alludes to changes in the environment. Perhaps DirectAdmin is not cut out to be a cargo ship control panel.

I'm not going to be a huge fan if DirectAdmin starts pushing out new updates every week or more often. There needs to be an LTS version of DirectAdmin that more or else stays stable - with security updates. But also include the ability to keep stack applications up to date separately.

At least that's my opinion, but I would encourage others to share their opinions.