FreeBSD 10.x support

I read through some of the custombuild scripting. The scripts appear to be written with RH/Centos as the default, with conditionals to handle the Debian branch, and FreeBSD. From what I read, clang should bring no changes whatsoever that impact DA. The problems come entirely from the new pkgng. You cannot just change all of the FreeBSD areas to the new syntax because you still need to support 9.x and older. pkg and pkgng are not compatible, so DA can’t install pkg. People are encouraged to move 9.x to pkgng, and it’s a one way trip. This means that DA doesn’t need a conditional within the FreeBSD conditionals for FreeBSD 10+, it needs one for pkgng. From what I read, pkgng works back to at least FreeBSD 8.3.

As far as clang is concerned, there is more to it now than GPL issues. I assumed that clang was a gcc clone to get around the GPL problems like they did with pf and carp. Not so. clang has a ton of advantages over gcc I learned from Googling. The industry is going to clang, not gcc, and not just for licensing advantages. Apple and Google are behind clang, and now Sony, despite some of the earlier benchmarks. clang has made every effort to be gcc compatible, but it appears as though gcc is going to have to evolve into a clang clone in order to remain relevant, and even then, they are at a licensing disadvantage as far as commercial software is concerned. Thus, I'm no longer concerned that clang will forever be the "red-headed step child".

PS: Personally, can't make up my mind whether I like GPL or BSD better. If I am the one sharing the code, I would want GPL 3. If I am the one making a product, I would want BSD. LOL!
 
Last edited:
A version with pkg instead of pkg_

Even if you still use GCC and you do not port DirectAdmin to 10.x yet, the issue with pkgng replacing old pkg_ system will be on 9.x series too:

I get this warning on a fresh installed server with FreeBSD 9.2:

make install clean
/!\ WARNING /!\
pkg_install EOL is scheduled for 2014-09-01. Please consider migrating to pkgng
http://blogs.freebsdish.org/portmgr/2014/02/03/time-to-bid-farewell-to-the-old-pkg_-tools/
If you do not want to see this message again set NO_WARNING_PKG_INSTALL_EOL=yes in your make.conf
 
meandrain said:
Even if you still use GCC and you do not port DirectAdmin to 10.x yet, the issue with pkgng replacing old pkg_ system will be on 9.x series too:

I get this warning on a fresh installed server with FreeBSD 9.2:

make install clean
/!\ WARNING /!\
pkg_install EOL is scheduled for 2014-09-01. Please consider migrating to pkgng
http://blogs.freebsdish.org/portmgr/2014/02/03/time-to-bid-farewell-to-the-old-pkg_-tools/
If you do not want to see this message again set NO_WARNING_PKG_INSTALL_EOL=yes in your make.conf
Click to expand...

The installation process on FreeBSD is completely messed up. You need to make sure CB doesn't run and uninstall all the packages DA installs before installing libs via ports and proceeding with CB (which has improved in the way it looks for dependencies).
 
interfasys said:
Portsbuild sound interesting :)
How do you inject all the extra required patches into ports? Via make.conf?
Click to expand...

Sorry for the late response! I'll make sure to enable forum notifications from now on. :)

I've never had to patch any ports to have them working with DirectAdmin. While I haven't tested every CustomBuild option out there, I have yet to come across a port that required manual patching to get it talking with DA. Port configuration will be done primarily through portmaster with a slew of arguments to auto-build & install the necessary software. I've also developed one-liner command-line installs that basically take care of everything needed on the system, e.g.

Code: 
portmaster -d lang/php5 www/apache24 mail/exim databases/mariadb55-server [...]

Can you please give me some examples of any patches that were required in your installs?

One of my primary goals is to maintain FreeBSD's standard of installation paths (such as keeping configuration files in /usr/local/etc) and change paths in directadmin.conf if possible to reference the corrected paths (or symlink when necessary). An example being DirectAdmin accessing Apache configuration files in /etc/httpd, which bothers me to no end. :) While there's no harm in doing so, my understanding is that system-level configuration files are kept in /etc, while third-party software is in /usr/local/etc. Another goal is to keep clutter down to a minimum with very little system modifications needed. Additionally, proper documentation is an absolute must with every function in the script properly explained. Finally, I want to ensure full compatibility with DirectAdmin, something of which will require a bit of work as future versions are released (and whether they contain any changes that break functionality).

interfasys said:
The installation process on FreeBSD is completely messed up. You need to make sure CB doesn't run and uninstall all the packages DA installs before installing libs via ports and proceeding with CB (which has improved in the way it looks for dependencies).
Click to expand...

Yup. To combat this problem, I will have to rewrite parts of DirectAdmin's setup.sh and offer an alternative script for us BSD'ers. I have yet to dive into setup.sh, but I am sure it can be done.

I'm going to be focusing more on PortsBuild coming this August, as I'll be taking some time off my regular job and diving deep into personal/open-source projects while I'm out of the country. I believe this effort is more important than ever, as the EOL of pkg_install is this upcoming September as previously noted.

A question for all: is it best to target PB for (new) 10.x installs first, with a focus on 9.x deployments later on? While 9.x is considered "Legacy", it doesn't mean it's not supported by FreeBSD. I see that 9.3-RELEASE has an ETA of July 2014, and I can see myself running 9.x for another year on some installs. There's also the question of converting existing CB+pkg_install installs to PB, which can be done with the proper steps. I currently run port-equivalents of pureftpd, ClamAV, SpamAssassin, PCRE, Perl modules (p5-) and other libraries (autoconf/automake/etc.) on production servers without any problems.

Man, I can't wait to get this project going! :D
 
mmx said:
A question for all: is it best to target PB for (new) 10.x installs first, with a focus on 9.x deployments later on?:D
Click to expand...
I would think so. The default install for 10 has pkgng out of the box, and there is no going back. 8.3-9.x have options for a DA install. I'd guess that if your conditionals are based on whether or not it is pkgng, rather than OS version, your code would fix 10.x, and those that migrate to pkgng from 8.3-9.x. The binaries are not the problem.
 
Last edited:
Our servers are still in a need to upgrade. If there's going to be support for FreeBSD 10 we'd gladly hop on that bandwagon. If not, or at least no time soon, we'll get 9.3 in 2 months.

So basically what I'm saying is, is there any idea of a date by which this will be available?
 
Hi gang, a quick update on my little PortsBuild project. I'm working on it during my spare time, but a lot has been done in terms of system design & documentation (more on that below).

I've noticed the latest CustomBuild supports pkgng, although I am not quite sure how much of the code is fully ported over. Regardless, I've converted 3 production servers (8.4, 9.1, soon 9.3) over from pkg_tools to pkgng for over 3 weeks now without issues. I recently used CustomBuild to update my systems to PHP 5.4.34 and MySQL 5.5.40, only to come across compiler troubles (had to use gcc48, as pointed out by @smtalk). Exim and Dovecot updated without hiccups.

Last month, while I was stuck on a plane going to Japan for 13 hours (and another 11 hours coming back) so I started writing PortsBuild by studying CustomBuild 2.0-RC7 and replacing its functions with one-liner portmaster commands. CB's 15,000 lines of code ended up getting reduced by 50%, with a lot of Linux-specific checks removed. That's when I realized @smtalk was in the middle of adding pkgng support to a few applications, and how much blood and sweat he puts into something thousands of DirectAdmin sysadmins use everyday. Furthermore, I went on to study setup.sh by DirectAdmin, used once for the installation of DA on a fresh system. This is where I plan PB to shine.

Two weeks ago, I realized PortsBuild is only really needed in two distinct situations: 1) first time DirectAdmin installation (hijack setup.sh calls to CustomBuild) and, 2) guide the sysadmin on updating Ports correctly. Of course, the third situation is when an admin decides to swap applications, e.g. Apache to nginx, which is quite involving to say the least (and something PB won't do initially, if at all). Now, regarding #2, an administrator can simply use portmaster to update their system if they wish to do so (which is what I do when updating ClamAV, SpamAssassin, libraries, PureFTPd, etc.) and only reference PortsBuild's documentation for special cases. PortsBuild does not necessarily need to handle your packages: 'portmaster' already does an excellent job in terms of package management in my opinion, so there is no need to reinvent the wheel.

Since I started PortsBuild for over a year ago, I've been documenting and learning what DirectAdmin and CustomBuild does when installed on a system. I've frequently scoured http://help.directadmin.com, and have religiously read every single DA changelog since I started using the software. :) I know where most configuration files end up and how applications need to be compiled, and have had tons of experience when things go wrong (and how to fix them). This brings us to PortsBuild's true purpose and design goal: to bridge the gap between DirectAdmin and FreeBSD+Ports. Essentially, I've written down the necessary steps involved to convert a system to Ports. An example I gave to a fellow DA user via PM was with Exim+ClamAV+SpamAssassin (off the top of my head, shortened):

  1. Do not follow these steps. Not tested for production use!
  2. # portmaster -d mail/exim mail/spamassassin security/clamav mail/sa-utils
  3. Enable certain features via config dialog, e.g. pyzor, razor-agents, etc.
  4. Symlink /etc/exim.conf to /usr/local/etc/exim.conf (for increased compatibility with DA, in case a future upgrade looks for the file there)
  5. Symlink SA binaries in /usr/bin/* to /usr/local/bin/*
  6. Symlink /etc/mail/spamassassin to /usr/local/etc/mail/spamassassin/
  7. Change path of sa-learn in SpamBlocker's exim.conf (/usr/bin/spamc => /usr/local/bin/spamc)
  8. Installation of p5 Perl modules for SpamAssassin runtime (e.g. p5-Net-Patricia, not auto-installed)
  9. Modify /usr/local/etc/clamd.conf as needed
  10. Modify /usr/local/etc/mail/spamassassin/local.cf
  11. Modify /etc/rc.conf and add:

    Code: 
    exim_enable="YES"
clamav_clamd_enable="YES"
clamav_freshclam_enable="YES"
spamd_enable="YES"
spamd_flags="-c -m 15"
  12. ... and so forth.

From what I can tell, DirectAdmin is ready to take on FreeBSD 10.x with nothing but Ports (my opinion). I've been running large Magento installs on FreeBSD 9.x/10.x machines with Apache (or nginx) with PHP-FPM and MariaDB 5.5 for years (without any control panels) so I am pretty confident interfacing DA is just a matter of having it access the configuration files in the right places by modifying 'directadmin.conf' and/or symlink as required. I am going to test a fresh new install of this type of setup (DirectAdmin, Apache, PHP-FPM, MariaDB, Exim, SpamAssassin, ClamAV and PureFTPd) in 2-3 weeks with a spare DA license I have, once I finalize hardware upgrades on a ESXi host. I'm hoping to have some good news soon.
 
mmx said:
That's when I realized @smtalk was in the middle of adding pkgng support to a few applications, and how much blood and sweat he puts into something thousands of DirectAdmin sysadmins use everyday.
Click to expand...

Thank you for your kind words! :) Do you know about a CB feature which allows you to disable management of specific component? (so that it wouldn't be listed in "./build versions" and places like that). Please check #29 of http://forum.directadmin.com/showthread.php?t=44743. It might be worth to disable specific components you manage in CB 2.0, so that they'd be controlled by your script. That way you could take control of SA+ClamAV only, for example.
 
smtalk said:
Thank you for your kind words! :) Do you know about a CB feature which allows you to disable management of specific component? (so that it wouldn't be listed in "./build versions" and places like that). Please check #29 of http://forum.directadmin.com/showthread.php?t=44743. It might be worth to disable specific components you manage in CB 2.0, so that they'd be controlled by your script. That way you could take control of SA+ClamAV only, for example.
Click to expand...

Thank you for making our lives easier! :D

I had no idea about #29, I may have skipped over that while going through your post. That will indeed make it much easier, thanks! Coincidentally, I've been wanting to squelch the PCRE message for quite some time. :)
 
Great work mmx!

Beside symlinks so that DA and CB know where to find config files, there are 2 things to be careful about
  • Users. Exim per example needs a different user than the default from ports)
  • Patches. You need to make sure patches from CB are renamed and stored in /files/. I know they exist for Dovecot and Exim per example.
 
Hi all, here's a quick update for those curious about the project. I've been dissecting setup.sh and have successfully "hijacked" the installation process of DirectAdmin and CustomBuild, going so far to modify/replace the scripts found in /usr/local/directadmin/scripts (where a dozen files are called one after the other on a new install, from update.tar.gz). I have documented every necessary step to setup the control panel on a fresh copy of FreeBSD 10.1 x64 with nothing but Ports. I'm in the middle of testing on my virtual machine while making great use of ESXi guest snapshots. :)

However, I'm currently at a roadblock right now, and it's in terms of copyright, licensing & distribution of the modified files. I am not sure if I am allowed to publicly release the modified setup.sh, install.sh, and other ones found in directadmin/scripts/ without the approval of JBMC Software, Inc. (or if I will even get authorization to do so). I contacted [email protected] about a week ago and have not received a reply since, which worries me that this project may not see the light of day in the end. Perhaps my request was not clear, or my email never made it through their servers. Regardless, I am ready to release a few scripts onto a GitHub repo and ask a few people to test, but I need permission from the DirectAdmin staff first to avoid any legal hurdles. Of course, none of the actual binaries will be released (specifically directadmin, dataskq, da-popb4smtp) only the various shell scripts that require modifications.
 
Hmm.. I don't recall seeing the E-Mail, but regardless we are interested in getting it out, in some form or another.
If the scripts are full rewrites, then by all means, you're welcome to release them.
But.. if they're only surgical changes, then perhaps a set of patches may work?

Maybe trying email again and I'll keep my eye out for in in our mail logs.
I'll look for the domain value set in the email for your forum username.
We can get a better idea on which direction to go once I see them :)

John
 
DirectAdmin Support said:
Hmm.. I don't recall seeing the E-Mail, but regardless we are interested in getting it out, in some form or another.
If the scripts are full rewrites, then by all means, you're welcome to release them.
But.. if they're only surgical changes, then perhaps a set of patches may work?

Maybe trying email again and I'll keep my eye out for in in our mail logs.
I'll look for the domain value set in the email for your forum username.
We can get a better idea on which direction to go once I see them :)

John
Click to expand...

Hi John, thanks for the reply!

I haven't made up my mind yet whether I want to release patches, full rewrites, a new script, or a mix of all three. I was hoping to get an okay to release the original (non-modified) scripts from update.tar.gz to simplify deployment. I want to make installation simple: "just download and overwrite" as opposed to "run this patch and replace these files" although most functions can be automated via scripting.

A lot of the files in scripts/ can be shrunk down by removing the Linux-specific code, however I am uncertain which way to proceed, as every scenario has both pros & cons.

I have a few installation scenarios and deployments in mind:
  1. Patch and Patch: patch setup.sh and patch files in scripts/ via one "patch.sh" file fetched separately. This allows both DirectAdmin+CB and PortsBuild code-bases to be similar and allows easier parallel development. Only the code for "patch.sh" will be made public, which is essential a file full of Perl/awk/grep/sed replacement routines that must be run in scripts/ after a modified setup.sh is executed. Downside for me is a larger code base to go through every time a major release is made.
  2. Patch and Re-Release: patch setup.sh, patch files in scripts/ and release a "revised" update.tar.gz. Like #1, it just makes distribution simpler for users. All of the contents of update.tar.gz will be made public (this is where I am asking for approval) minus the compiled binaries.
  3. Patch, Full Replace and Re-Release: patch setup.sh, patch files in scripts/, replace or remove unneeded scripts, and release a "revised" update.tar.gz (like #2). This branches off into its own code-base (with Linux-specific code removed) yet follows the same style of scripting and function-calling of the original update.tar.gz.
  4. Patch and New Release: patch setup.sh, call a new script "portsbuild.sh" to auto-install all libs, deps and services from ports. This is branching off from install.sh and CustomBuild entirely. It will replace update.tar.gz completely, minus the compiled binaries. Existing code from the tarball will be used, hence authorization & permission is still required to distribute the original files or re-use functions written by DirectAdmin. This is where I ultimately see myself going.

I'll resend the email once more. I can show you some examples once I clean up the code. :) Thanks!
 
Hello
I want to run directadmin on FreeBSD 10.1 (amd64), we know when FreeBSD 10 will be officially supported ?

Regards
MJ
 
mmx said:
Hi John, thanks for the reply!

I haven't made up my mind yet whether I want to release patches, full rewrites, a new script, or a mix of all three. I was hoping to get an okay to release the original (non-modified) scripts from update.tar.gz to simplify deployment. I want to make installation simple: "just download and overwrite" as opposed to "run this patch and replace these files" although most functions can be automated via scripting.

A lot of the files in scripts/ can be shrunk down by removing the Linux-specific code, however I am uncertain which way to proceed, as every scenario has both pros & cons.

I have a few installation scenarios and deployments in mind:
  1. Patch and Patch: patch setup.sh and patch files in scripts/ via one "patch.sh" file fetched separately. This allows both DirectAdmin+CB and PortsBuild code-bases to be similar and allows easier parallel development. Only the code for "patch.sh" will be made public, which is essential a file full of Perl/awk/grep/sed replacement routines that must be run in scripts/ after a modified setup.sh is executed. Downside for me is a larger code base to go through every time a major release is made.
  2. Patch and Re-Release: patch setup.sh, patch files in scripts/ and release a "revised" update.tar.gz. Like #1, it just makes distribution simpler for users. All of the contents of update.tar.gz will be made public (this is where I am asking for approval) minus the compiled binaries.
  3. Patch, Full Replace and Re-Release: patch setup.sh, patch files in scripts/, replace or remove unneeded scripts, and release a "revised" update.tar.gz (like #2). This branches off into its own code-base (with Linux-specific code removed) yet follows the same style of scripting and function-calling of the original update.tar.gz.
  4. Patch and New Release: patch setup.sh, call a new script "portsbuild.sh" to auto-install all libs, deps and services from ports. This is branching off from install.sh and CustomBuild entirely. It will replace update.tar.gz completely, minus the compiled binaries. Existing code from the tarball will be used, hence authorization & permission is still required to distribute the original files or re-use functions written by DirectAdmin. This is where I ultimately see myself going.

I'll resend the email once more. I can show you some examples once I clean up the code. :) Thanks!
Click to expand...

Awesome work mmx! Do you happen to know if DirectAdmin is going to support your packages?
I'm currently working on 9.3 but upgrading to 10.x would be nice :).
 
You must log in or register to reply here.
Back
Top