FreeBSD 10.x support

IT_Architect

Verified User
Joined
Feb 27, 2006
Messages
888
Awesome work mmx! Do you happen to know if DirectAdmin is going to support your packages? I'm currently working on 9.3 but upgrading to 10.x would be nice :).
I'd rather not wait for that. Why would DA care? It helps them sell DA, and they don't have to support the install if someone else writes it, derivative or not. Let MMX prototype everything, and if it works, DA may want to pay him, or steal some of his ideas. Otherwise, let MMX sell the modified routines. DA can't lose either way. They still own DA.
 

mmx

Verified User
Joined
May 8, 2005
Messages
112
Location
Montreal, QC, Canada
Awesome work mmx! Do you happen to know if DirectAdmin is going to support your packages?
I'm currently working on 9.3 but upgrading to 10.x would be nice :).
No idea. :)

I'd rather not wait for that. Why would DA care? It helps them sell DA, and they don't have to support the install if someone else writes it, derivative or not. Let MMX prototype everything, and if it works, DA may want to pay him, or steal some of his ideas. Otherwise, let MMX sell the modified routines. DA can't lose either way. They still own DA.
As much as I want to make money, there's nothing proprietary about what I've done, so I'm going to release my work to the wild (open source). I'm thinking of offering paid support down the line, but it all depends how my scripts work out.

Speaking of which, after taking a 9 month "break" (as in, other projects got in the way) I'm less than a month away to releasing PortsBuild. These past 3 days, I spent another 20 hours and rewrote a new method/script from scratch. Here is what I've accomplished:

  • Everything installed using FreeBSD pkg (packages) except Exim from ports (to enable SPF, DMARC and DKIM support). Majordomo was the only package I wasn't able to install from ports (broken for over a year) so I used DirectAdmin's version (majordomo.sh).
  • Move majority of configuration files from /etc/ to /usr/local/etc to stay compliant with FreeBSD's hier(7). Only files I haven't decided to store are SSL certificates for Apache, Exim and Dovecot. Do I keep them separate (e.g. /usr/local/etc/apache24/ssl) or move them into /usr/local/etc/ssl (or even /etc/ssl)?
  • Have PortsBuild use CustomBuild to rewrite configuration files (e.g. ./build rewrite_confs) for DirectAdmin compatibility along custom configuration files (e.g. custom/ap2/) to reuse existing templates.
  • Avoid having to use setup.sh from the beginning, but I did have to use some of DirectAdmin's code to download the license file & binaries from their server (so I still need to ask their blessing to avoid lawsuits).

Here's the services I have supported thus far (with more coming):
  • Apache 2.4.17
  • PHP-FPM 5.6.14
  • MariaDB 5.5
  • Exim 4.86 + ESF + BC (latter two untested, still new to me)
  • Dovecot 2.2.19
  • SpamAssassin 3.4.1
  • ClamAV 0.98.7
  • ProFTPd 1.3.5

Roundcube is in progress. Looking at CB2's code, it looks fairly involving to have it tie in with the rest of the system. Speaking of which, shoutouts to @smtalk for the refactored version of CB2. Really nicely done.

Limitations of PortsBuild so far:
  1. Can only run one version & instance of PHP. I haven't researched any methods or solutions yet.

What's left to do:
  1. Fix a few minor bugs.
  2. Clean up the code.
  3. Make it easy to configure and use + add safety nets.
  4. Polish up documentation.
  5. Support additional services (after releasing).
  6. Test on FreeBSD 10.2 x64.
 

IT_Architect

Verified User
Joined
Feb 27, 2006
Messages
888
As much as I want to make money, there's nothing proprietary about what I've done, so I'm going to release my work to the wild (open source). I'm thinking of offering paid support down the line, but it all depends how my scripts work out.
Money, in and of itself, is proven to be a poor motivator. Your creating this script proves this is simply something you wanted to do, and DA not creating a script proves it is something they don't want to do even though it is profitable for them to do so. However, money is an enabler for you to do what you want to do.

If you wrote the script, it is proprietary. If you write a patch for their script, then your patch is proprietary, open source or not. It is impossible for DA to put you in a box for selling what is yours. You could even sell subscriptions for your patch per license and there are a thousand ways to protect it if you want to. You would be enabled, and FreeBSD users of DA would have a script to keep their servers up to date. That isn't happening now. There will always be those who hack your output to not pay, but most people would be glad to pool with others to pay you to do it because they are smart enough to figure out they would be getting paid less than a Chinese factory worker if they have to hack your output every time FreeBSD or DA makes a change.

If you want to make it an open source project, it is common for people to donate code with the understanding they are giving up rights to it. You can be the Linus Torvad of the project, and provide the point of contact for the official version. You could even maintain a port for it.

If you can deliver, I personally believe that DA would be silly not to give someone like you 1/3 of the licensing fees collected from their FreeBSD customers, to support DA under FreeBSD for them. FreeBSD has long been the "red-headed step child" at DA. Lately, DA has been losing the whole FreeBSD ballgame through lack of support, so they would have nothing to lose and revenue to gain, and you would be enabled.
 
Last edited:

labrocca

Verified User
Joined
Mar 12, 2006
Messages
151
After 15 years of BSD usage I'm considering going to CentOS and that's almost exclusively due to DA not supporting the 10x branch. Because I have no qualms about FreeBSD 10 and actually think it's a great OS. Sadly DA doesn't seem to want to step up their game and catch up their support for it.

Even worse for me personally I just spent a lot of time setting up a new production setup only to run into some roadblocks with DA+BSD10. So now I may need to spend hundreds of dollars for a new server, spend countless hours on new setup, and experience a day of downtime to potentially move.

I am rather frustrated at all this. I would hope that by listing FreeBSD being supported they'd actually support the latest versions. Production branches of BSD are 9.3, 10.1, and 10.2.

Please DA, continue to support FreeBSD. It's the main reason I use DA and I feel as though I'm being forced to choose between DA and BSD. Seems unfair.
 

mmx

Verified User
Joined
May 8, 2005
Messages
112
Location
Montreal, QC, Canada
After 15 years of BSD usage I'm considering going to CentOS and that's almost exclusively due to DA not supporting the 10x branch. Because I have no qualms about FreeBSD 10 and actually think it's a great OS. Sadly DA doesn't seem to want to step up their game and catch up their support for it.

Even worse for me personally I just spent a lot of time setting up a new production setup only to run into some roadblocks with DA+BSD10. So now I may need to spend hundreds of dollars for a new server, spend countless hours on new setup, and experience a day of downtime to potentially move.

I am rather frustrated at all this. I would hope that by listing FreeBSD being supported they'd actually support the latest versions. Production branches of BSD are 9.3, 10.1, and 10.2.

Please DA, continue to support FreeBSD. It's the main reason I use DA and I feel as though I'm being forced to choose between DA and BSD. Seems unfair.
Hi, what kind of troubles did you have with FreeBSD 10.x? Perhaps I can show you how to fix it. :)

FYI, I am running DirectAdmin with everything installed from Ports on a production system on 9.3, and currently testing 10.2. There are a few issues to iron out (unrelated to FreeBSD) but overall the system works.
 

IT_Architect

Verified User
Joined
Feb 27, 2006
Messages
888
Hi, what kind of troubles did you have with FreeBSD 10.x? Perhaps I can show you how to fix it. :)
A good "How To" that works and a description of the limitations might be a start. I run VMware's VSphere as my free operating system with web servers as VMs. The cure for me will not be DA, Linux, and DLL hell. The cure will be Windows, where I can run both WAMP and Windows web hosting software on the same server. I already have a 2012 R2 VM that I recently set up. When a guy like you can make it happen and DA can't, that speaks volumes about DA's strength and market share.
 

labrocca

Verified User
Joined
Mar 12, 2006
Messages
151
I have issue installing php-fpm55. Says this:

Code:
Starting php-fpm55: /usr/local/php55/sbin/php-fpm55: error: `/usr/local/php55/sbin/.libs/php-fpm' does not exist
 This script is just a wrapper for php-fpm.
And checking after the install the php-fpm55 file is indeed a wrapper script and I can't find the binary if it's installed at all. Same happens if I try to go with php-fpm56.

Also upgraded openssl from pkg and after that Nginx wouldn't build at all. I had to uninstall openssl port, which of course screwed up other builds and I had to reinstall the pkg openssl and rebuild apache, php-fpm54, and just about everything else except for Nginx which now won't build.

So you see, I'm dealing with a bit of a mess and I'm forced to run a test environment to get this sorted. I've been working with my production server and it's had to go offline for hours while I attempt to fix these issues. Which so far I've had no real luck doing.
 

mmx

Verified User
Joined
May 8, 2005
Messages
112
Location
Montreal, QC, Canada
A good "How To" that works and a description of the limitations might be a start. I run VMware's VSphere as my free operating system with web servers as VMs. The cure for me will not be DA, Linux, and DLL hell. The cure will be Windows, where I can run both WAMP and Windows web hosting software on the same server. I already have a 2012 R2 VM that I recently set up. When a guy like you can make it happen and DA can't, that speaks volumes about DA's strength and market share.
I've been tempted to just release the how-to first for those interested in setting up the system manually versus the scripted install (you don't even need to download/execute DirectAdmin's setup.sh anymore). I've already written the steps line by line with plenty of comments, but it needs a bit of cleaning up. I'll go ahead and release the how-to to get the ball rolling as I don't want to disappoint the community. I'll put it up on http://github.com/portsbuild very soon.

I've posted a list and a request in the forums to have the current limitations fixed.

I hate to say it, but it's true: DA has somehow lagged behind FreeBSD support since the early 9.x days, even though FreeBSD is arguably the easiest system to work with. The issue stems from having CB target all the various kind of Linux distributions out there first and leaving FreeBSD as the last operating system to work on. I have much respect for @smtalk and the work that he's done with CB; it's possible he is a lot more comfortable with Linux systems rather than FreeBSD ones. Even though Linux and BSD "function" the same way, the underlying principles and philosophies of BSD tend to induce "fear" in majority of Linux users. :D

You'd laugh if I told you I was able to shrink CustomBuild to 100-200 lines of code if I was only installing onto a FreeBSD system with nothing but Ports. :)
 

mmx

Verified User
Joined
May 8, 2005
Messages
112
Location
Montreal, QC, Canada
I have issue installing php-fpm55. Says this:

Code:
Starting php-fpm55: /usr/local/php55/sbin/php-fpm55: error: `/usr/local/php55/sbin/.libs/php-fpm' does not exist
 This script is just a wrapper for php-fpm.
And checking after the install the php-fpm55 file is indeed a wrapper script and I can't find the binary if it's installed at all. Same happens if I try to go with php-fpm56.

Also upgraded openssl from pkg and after that Nginx wouldn't build at all. I had to uninstall openssl port, which of course screwed up other builds and I had to reinstall the pkg openssl and rebuild apache, php-fpm54, and just about everything else except for Nginx which now won't build.

So you see, I'm dealing with a bit of a mess and I'm forced to run a test environment to get this sorted. I've been working with my production server and it's had to go offline for hours while I attempt to fix these issues. Which so far I've had no real luck doing.
For starters, the installation prefix looks wrong. How did you install PHP 5.5/5.6? Did you use CustomBuild2 or lang/php55|56?

If you are running a recent version of FreeBSD, you rarely need to touch OpenSSL from pkg/ports. It'll cause more headaches than it's worth, trust me (which you do, since nginx failed to build). I recommend sticking with the base OpenSSL libraries, as long as you are running a current version of FreeBSD.

Now, the solutions.

If you are installing everything from CustomBuild2, then stick with the base OpenSSL libraries and do a ./build all d in /usr/local/directadmin/custombuild (this is untested, however).

If you installed everything from pkg/Ports, do the following:

Code:
# Comment out any changes done to /etc/make.conf

# Remove the OpenSSL pkg
pkg delete -f openssl

# or:
cd /usr/ports/security/openssl && make uninstall && make clean

# Reinstall/rebuild all packages, either
# via pkg (quicker, but might break configured Ports):
pkg upgrade -f

# or via portmaster (longer and safer):
portmaster -a -f -d
 

labrocca

Verified User
Joined
Mar 12, 2006
Messages
151
You'd laugh if I told you I was able to shrink CustomBuild to 100-200 lines of code if I was only installing onto a FreeBSD system with nothing but Ports.
Not really laughing and I 100% believe you. I was never sure why DA didn't take more advantage of ports to begin with and then just create custom configs so that each service was setup and ran smoothly.

I've been using DA for a decade. Seemed to always have good BSD support up till about 2-3 years ago imho. They took a while to support version 9 and I get the feeling they don't want to support version 10. There isn't any reason they can't make it happen that I can see.

For starters, the installation prefix looks wrong. How did you install PHP 5.5/5.6? Did you use CustomBuild2 or lang/php55|56?
Just did CB2 php-fpm55. I know better than to use ports if there is a CB package.

If you are installing everything from CustomBuild2, then stick with the base OpenSSL libraries and do a ./build all d in /usr/local/directadmin/custombuild (this is untested, however).
Just have to make sure my custom changes are all saved and part of a ./build all d before I do it.

If you are running a recent version of FreeBSD, you rarely need to touch OpenSSL from pkg/ports.
I think a pkg audit told me to upgrade the openssl. Almost certain I didn't install it myself unless it's part of the recommendation from DA. Doesn't look like it is...
http://help.directadmin.com/item.php?id=354 One of those may have needed it though for a dependency.

I have countless hours into this server and configuration. It's all falling apart very quickly too. And while today, right now, it's all working...I can't build Nginx or upgrade php. That's less than optimal and is going to represent a problem when I need to do so. When I run ./build package it should always install.

btw mmx thanks for your help.
 

mmx

Verified User
Joined
May 8, 2005
Messages
112
Location
Montreal, QC, Canada
Not really laughing and I 100% believe you. I was never sure why DA didn't take more advantage of ports to begin with and then just create custom configs so that each service was setup and ran smoothly.

I've been using DA for a decade. Seemed to always have good BSD support up till about 2-3 years ago imho. They took a while to support version 9 and I get the feeling they don't want to support version 10. There isn't any reason they can't make it happen that I can see.



Just did CB2 php-fpm55. I know better than to use ports if there is a CB package.



Just have to make sure my custom changes are all saved and part of a ./build all d before I do it.



I think a pkg audit told me to upgrade the openssl. Almost certain I didn't install it myself unless it's part of the recommendation from DA. Doesn't look like it is...
http://help.directadmin.com/item.php?id=354 One of those may have needed it though for a dependency.

I have countless hours into this server and configuration. It's all falling apart very quickly too. And while today, right now, it's all working...I can't build Nginx or upgrade php. That's less than optimal and is going to represent a problem when I need to do so. When I run ./build package it should always install.

btw mmx thanks for your help.
No problem!

As long as you kept your CB2 customizations in the custom/ directory, then you can safely do a ./build all d without worry. If you can disclose some of the customizations you've done, I might have a better idea as to what's breaking.

There are some Ports that give you the option to install OpenSSL from Ports as well, but generally the default selection is to use the libraries from the base system. DirectAdmin itself and its dependencies never install OpenSSL via pkg or Ports.

Which reminds me: did you modify the SSL configuration paths in CB2's configure scripts? e.g. for Apache, leave "--with-openssl " as-is without defining the path to the libraries. Same advice goes for nginx ("--with-http_ssl_module") and PHP ("--with-openssl "). One thing I'll recommend is to remove any references to Kerberos (e.g. " --with-kerberos ") to minimize the amount of crap that gets installed with it. Also, turn off GSSAPI support in Curl and other services that ask for it.

Expected downtime to remove OpenSSL and rebuild all ports is very little; I'm sure most customers won't notice it after hours or late at night. I usually do rebuilds after updating OpenSSL in the following order, prioritizing email services first:

  1. # Rebuild pkgs/Ports that rely on OpenSSL, e.g. Perl, wget, Python, etc. then rebuild services via CB2 (example)
  2. ./build exim
  3. ./build dovecot
  4. ./build spamassassin (if you've installed via CB2)
  5. ./build apache (or nginx)
  6. ./build mysql
  7. ./build php d

Another tip I can share is to use devel/ccache to speed up re-compiles dramatically, but this might only work for Ports.
 

Vibe

Verified User
Joined
Aug 3, 2005
Messages
111
MMX - you are to be highly commended! Your diligence and hard work is greatly appreciated, and opens the door for users that rely on FreeBSD + DirectAdmin. I have been following this thread for quite a while after learning of the possibility that FreeBSD may no longer be supported. I have been using DirectAdmin with FreeBSD since 4.X and it has been a wonderful experience. The new pkg system has been great to work with and I can't imagine that such drastic changes as seen in FreeBSD 10.X will be the norm for development in the future. With your installation script I can only hope that DirectAdmin takes notice of your request to modify binary names + paths in order to offer continued support for their loyal FreeBSD users. Hats off to you MMX - I look forward to trying your FreeBSD PortsBuild Script in the very near future. This is fantastic news!
 
Last edited:

mmx

Verified User
Joined
May 8, 2005
Messages
112
Location
Montreal, QC, Canada
MMX - you are to be highly commended! Your diligence and hard work is greatly appreciated, and opens the door for users that rely on FreeBSD + DirectAdmin. I have been following this thread for quite a while after learning of the possibility that FreeBSD may no longer be supported. I have been using DirectAdmin with FreeBSD since 4.X and it has been a wonderful experience. The new pkg system has been great to work with and I can't imagine that such drastic changes as seen in FreeBSD 10.X will be the norm for development in the future. With your installation script I can only hope that DirectAdmin takes notice of your request to modify binary names + paths in order to offer continued support for their loyal FreeBSD users. Hats off to you MMX - I look forward to trying your FreeBSD PortsBuild Script in the very near future. This is fantastic news!
Thanks for the support. :)

Looks like no response from the DA staff regarding my request thread (oh well). Perhaps they're waiting to see if I actually release my promised script before they start implementing the requests. In any case, a work in progress version of PortsBuild has been up for a month or so. It's not actually functional right now, but one can use the guide to setup a system manually in the mean time (if they're crazy to do so). :)

I've come up with some alternatives to fix the issues I had outlined in the thread for now (basically restarting the services using the post-function scripts when adding a new user in DA).
 

marcvh

New member
Joined
Sep 29, 2009
Messages
4
Any word on official FreeBSD 10 support?

FreeBSD 9.3 has an End of Life date of December 31st 2016, so we will have to upgrade before that date. If Directadmin chooses not to support FreeBSD 10 we might have to start looking at alternatives.
 

Chrysalis

Verified User
Joined
Aug 25, 2004
Messages
1,584
Location
uk
Will email John to ask whats up. I think we may have just thrown too much at him.

The priority is to just get 10.x binaries compiled for DA, and the existing CB scripts should work providing a gcc port is compiled from ports.

Then after that is done maybe some work can be started on a more BSD friendly CB. I agree with Olivier that gcc is the better way to go than trying to use clang, because we have to remember CB is designed to be compatible with all distros and it makes sense to use the same compiler as all the other distros.

Meanwhile the 9.x version of DA should work but with needing the compat9x libraries installed on the system.
 

mmx

Verified User
Joined
May 8, 2005
Messages
112
Location
Montreal, QC, Canada
Will email John to ask whats up. I think we may have just thrown too much at him.

The priority is to just get 10.x binaries compiled for DA, and the existing CB scripts should work providing a gcc port is compiled from ports.

Then after that is done maybe some work can be started on a more BSD friendly CB. I agree with Olivier that gcc is the better way to go than trying to use clang, because we have to remember CB is designed to be compatible with all distros and it makes sense to use the same compiler as all the other distros.

Meanwhile the 9.x version of DA should work but with needing the compat9x libraries installed on the system.
DirectAdmin runs out of the box on 10.3 x64 with compat9x. There's no immediate need to recompile it as far as I know. I've had my test system run on FreeBSD 10.3 for a while now with everything installed from ports (yes, I skipped on using CB).
 

ReN

Verified User
Joined
Jul 2, 2005
Messages
185
Yeah i wanted to try stick with the DA ./build scripts as much as possible - from the testing i did , there was a GCC issue as the default for 10.x i believe is "clang", simple fix via /etc/make.conf , the other issue was the perl version used, from memory everything else seemed to ok ish.... but , with 3 months until the EOL of 9.3 things need to be moved along i'm thinking.
 

Chrysalis

Verified User
Joined
Aug 25, 2004
Messages
1,584
Location
uk
I think this has become higher priority, the recent ssl problem I discovered I think is directly linked to the fact that the FreeBSD 9 binaries are compiled on 9.1 which has obsolete ssl libraries.
 
Top