cleaner builds and using RPMs instead of custom builds

empowering

empowering

Verified User
Joined
Aug 2, 2005
Messages
167
Location
New York
Hi,

Let me first state, I love you guys at DA and the software.

My biggest gripe about your software management. You compile way too much software or create custom rpms (that the source rpms don't complie out of the box in this case I'm specifically referring to exim)

Let me give you some specific examples (at least with CentOS/RHEL):
- exim, why create a custom rpm? why not use a default rpm and then tack on custom patches OR your own files
- the da_exim rpm itself you created can't even clean install (a conflict with setup rpm) you must force to install. A bad practice, hard to then automate upgrades
-dovecot, why is this complied? why isn't it a rpm?

While not as bad as cpanel, you are doing WAY too much custom compling. While it's needed on systems that don't have package managers, you create a maintainance nightmare when managing many machines that do have them.

Many of the distros you support do have package managers. You should be using as much as default built into the distro or if not at least create your own packages instead of compiling everytime a new version comes out.

The only legit apps I see could be compiled are apache and php.
 
Last edited:
No one seems to care how hard to maintain an existing DA server once installed??? Everyone here manually does updates per server???

We are looking to ensure all servers/VPSes we deploy are secure and update system software. IMHO right now the scripts they have don't cut it.
 
Hello,

1) we create custom exim rpms because we need perl.o compiled in. It doesn't (or never used to) come in stock rpms.

2) I can look into removing /etc/aliases from the config list of the rpm for the setup rpm conflict. Exim does use it, but it's easy enough to not include in the list and the rpm system wouldn't know.

3) Because we support about 41 OS versions now and there are frequent version updates. It would take us eons to create binaries for each OS each time there is a new version released. As it stands now, we simply download the tar.gz to our files server and change a version number and we're done. This saves us an absolutely immense amount of time allowing us to do other things. Compiling is also OS independant and also ensure's that the binaries created will always match up to your libraries it links to, each time. Even though systems may be the same version, it's common that some have newer openssl versions, etc, so the binaries don't always match up. We compile as static when we can to avoid this, but many system don't allow that anymore if linking to system authentication libraries, forcing dynamic binaries, and linking errors for modified OSs.

John
 
No one seems to care how hard to maintain an existing DA server once installed???
Click to expand...

That's because its not that hard. There are very few things you have to use custom DA stuff for anyway.

Apache
mod_ whatever for apache
Exim
MySQL
php

That's about all. I maintain 70 servers by myself with plenty of time to spare. Maybe you have a lot more servers. How hard a company is to maintain is a relative thing.
 
DirectAdmin Support said:
Hello,

1) we create custom exim rpms because we need perl.o compiled in. It doesn't (or never used to) come in stock rpms.
Click to expand...

Which is a valid reason to create a custom rpm.

2) I can look into removing /etc/aliases from the config list of the rpm for the setup rpm conflict. Exim does use it, but it's easy enough to not include in the list and the rpm system wouldn't know.
Click to expand...

Yes if it's in conflict with another required RPM that's the best way of handling it. If using your methodology, then the alias should be installed via your script, not in the RPM.

3) Because we support about 41 OS versions now and there are frequent version updates. It would take us eons to create binaries for each OS each time there is a new version released. As it stands now, we simply download the tar.gz to our files server and change a version number and we're done.
Click to expand...

If they were using package management, no in most cases it's just changing the version # and rebuilding the package. No different than you describe. I donno how automated/manual your development process is so I can't speak for how you do it, but it's all definitely possible with not much extra work.

Out of the the OSes I'm aware you support, FreeBSD is the only one that doesn't support a package manager. CentOS, Fedora, RHEL, Debian all have package managers. Hell with most using RPMs you could even take a SRPM created and compile it on the destination machine and then install.

This saves us an absolutely immense amount of time allowing us to do other things.
Click to expand...

In mosts cases the distro already has all of the applications you need. If not there are already reliable packages out there available to do the job. If I'm mistaken there isn't many custom patches that DA makes to each of the complied source code?

The difference is if someone wants something newer than their distro, leave that to the sysadmin to decide how they want to do it, not DA. IMHO DA should be working closer with the distro they offer, not against them. By custom compiling code that's exactly what DA's doing and in the end makes server harder to maintain. Would you agree with this statement?

I'm not saying using everything already built into the distro BUT PLEASE use their package manager as much as possible.

Ok let me ask you this, how does DA recommend keeping all servers updated with the custom complied code? Manually SSHing into each machine?

You have the installing/creating a new DA instance down pat. How does DA keep them updated? This is where DA's software falls short, and most of the discussions in the forum is about this.
 
floyd said:
That's because its not that hard. There are very few things you have to use custom DA stuff for anyway.

Apache
mod_ whatever for apache
Exim
MySQL
php

That's about all. I maintain 70 servers by myself with plenty of time to spare. Maybe you have a lot more servers. How hard a company is to maintain is a relative thing.
Click to expand...

You actually make it much simpler than it is. I'm sure hostpc.com would agree with me on this and would be curious what his comments on this would be since he has a larger DA install than us.

You manually SSH into each machine? IMHO that's an outdated methodology. We use Puppet (http://reductivelabs.com/projects/puppet/), which works GREAT by the way...

My point is, everything else is managed automatically with Puppet, EXCEPT DirectAdmin's many custom complied options. Yes you mention the top level features, but some (ie php) have 10-15 additional compiles.

My overall point to this thread...
- custom complies with distros that have package management makes it harder to maintain, not easier for the sysadmin
- custom complies can make the system less secure (because you have to remember manually install the latest update) All it takes is missing one machine. There is no simple automatic method to know you have the latest scripts installed.
- while DA has the install scripts down pat, how do you ensure they are updated properly?
 
You actually make it much simpler than it is.
Click to expand...

Then I must be a freaking genius along with a bunch of other people here.

You have made your request and DA has answered.
 
Ok, since yall asked, I'll toss in my 2cents. :)

I agree 100% with Larry about the complexities of compiling multiple DA servers across multiple platforms. DA seems to make this harder than it really has to be. Larry has excellent points on creating RPM based updates and keeping them updated regularly.

We maintain several hundred servers in multiple datacenters and I gotta admit, it's a nightmare keeping track of updates. Most notably, Exim/Dovecot as Larry suggests. Puppet is a great tool (his link is wrong, it should be http://reductivelabs.com/trac/puppet) - but when you're working across multiple platforms, it can still get hairy. We've got a mix of everything from FreeBSD, Redhat9, Fedora and RHEL - keeping all packages updated 24x7 is a full time job. Right now, we're fighting a dependency issue nobody can seem to solve which is affecting our Yum routines - but that's another battle for another day.

One large issue I've got with the Exim distributions is the lack of native Dovecot. IMHO, Directadmin should get rid of the vm-pop3d daemons/routines and convert everything to Dovecot starting with the next DA update. vm-pop3d is severely outdated and needs to be abandoned. Make one mail distribution - Dovecot enabled. Again, as Larry mentioned, one RPM repository would fix it.

50% of the beef I've got with DA is there should be one update script that goes out, updates server software, then completes with a full DA upgrade based on server OS, installed modules, Apache versions, etc. One update to take care of everything. No third party package (puppet etc) will handle all that, so yes, we do resort to SSH for a lot of tasks, however we've automated scripts, by server grouping, to login to each machine, perform the task, then move to the next machine without user interaction. DA RPM's would solve this issue.

Another major issue I've got with DA is the number of people taking over projects that are basically core to DA. For instance (and this is NO offense intended to anyone) - some people are creating Exim distributions "on the side" - others are creating and "fixing" apache install routines - there's TOO MANY hands in the pot - with no central quality control. Rarely do I use the fixes by third parties until they're released "officially" by Directadmin. Sometimes the fixes are good, sometimes I find they're not, but nearly ALL the time I find that the QC that is performed could be better, and should be done by DA, not by users.

We've been waiting for months for "fixes" to the "jail shell" that Mark released on a "here ya go, not sure if it's ready" basis more than a year ago (or was it two?). We were able to work with that release and "make it work" - but it immediately broke with apache2.

I don't know how many people are currently working at DA - or at keeping it updated specifically, but it seems to me that there should be enough incoming revenue to have someone dedicated to providing updates and the rest of the company working on new features, sales, support, etc.

Overall, Directadmin is an excellent product and we all prove that every day by putting our company reputation into the promotion, sale and support of this product. There are however several points that could make things a LOT easier for us, as administrators - including an RPM system that works.

Thanks Larry for bringing this thread to my attention - and for starting a conversation that was needed, but perhaps we all were too busy trying to avoid.

Joe
 
I'm in agreement with hostpc.com and Larry, updating the software components of DirectAdmin can be a pain and is probably difficult to automate. Install this RPM for this software (Exim), run the build script to update other software (Apache, PHP, etc.), and run Yum to update the software for the system. Gosh, it would be nice if I could run one command to update everything that's installed. :)
 
floyd said:
That's because its not that hard. There are very few things you have to use custom DA stuff for anyway.

Apache
mod_ whatever for apache
Exim
MySQL
php

That's about all. I maintain 70 servers by myself with plenty of time to spare. Maybe you have a lot more servers. How hard a company is to maintain is a relative thing.
Click to expand...

Well, aint you special? :)

I'm glad your DA installs are secure - now what about your server software / OS / Kernel?
 
And I agree hundreds of servers are going to be a lot harder than my 70.
 
Joe I am not going to get drawn into an argument with you. I have always respected you. You have your opinion of what is hard to maintain and I have mine.

I personally think DA does a good job of maintaining the software it installs. It cannot be expected to maintain all the other server software and OS and kernel updates since it did not install them in the first place. It can only update the software it installed. It is not meant to take the place of yum or apt-get.

Yes it is my opinion. My opinion has worked out very well for me. Your mileage may vary.
 
Indeed, I don't get into arguments, just discussions with adult minded collegues, of which I consider you and Larry.

Blindly running yum -y update is bad practice, IMHO - especially given RHEL OS and DA's propensity to change the exclude line in most updates. If one package installs that shouldn't - you won't know it till after the update completes - and by then, it'll take you hours to recover from the mistake.

(for those that don't know what excludes I'm talkinig about, the following should be excluded in your /etc/yum.conf (RHEL systems)
exclude=apache* httpd* mod_* mysql* MySQL* da_* *ftp* exim* sendmail* php* bind* named* )

The topic of this thread is to persuade and give reasonable, rationable arguments to using the OS package managers (there are several) to handle known upgrades. That way, the OS can do the work for us - simply by rpm -V'ing the needed upgrades globally - rather than per individual servers. Forcing rpm's to install is bad practice, IMHO. Allowing yum or apt to globally and unilaterally install new packages, kernels and mods is bad practice, IMHO.

Larry's suggestions are dead on. Your suggestions are dead on. My suggestions (IMHO) are dead on. All three, together, are needed to successfully run DA, insure stable servers, and the ultimate goal, provide the best user experience for our cumulative customers - because someday, your customers might be mine, and my customers might be yours. We need to work together to insure we're all providing the same stable environment - to continue to show customers the benefits and security of Directadmin-v-cPanel and to continue to grow our profits.

Good luck to all involved in this process.
 
Oh forgot this...

hostpc.com said:
exclude=apache* httpd* mod_* mysql* MySQL* da_* *ftp* exim* sendmail* php* bind* named* )
Click to expand...

Oh thanks for bring this up Joe. :-) Another part of DA's setup I have issue with. You can't have your own yum repo and install updates for MySQL (mysql.com's versions), DA's exim, proftpd, or named because of this line. If DA removes the offending RPMs, why exclude them? Yum by default will not update these RPMs unless they are newer than what's available or already installed. In addition I assume most sysadmins do not have a crontab with yum to automatically update their scripts. I also assume most sysadmin's do not use third party repos like DAG so this shouldn't be an issue. I understand the logic of why DA would put this here (to prevent less support tickets for them) If DA had their own yum repo this would be a one stop update.

We've wound up having puppet force replacing yum.conf with our own, so these CAN be updated with our repo RPMs.

With puppet, while our updates are automatic for specific RPMs , we do test and develop recipes first. We also break out yum and puppet into a seperate syslog file to ensure things are installed correctly. Things aren't just blindly installed.
 
Blindly running yum -y update is bad practice
Click to expand...

Ok I am not saying you are wrong. I am here to learn too.

I have been running yum -y update for 3 years on DA servers with Fedora and CentOS and have not had a problem yet. I use the exclude line as well. I have never had the exclude line change that I know of.

But even if the exclude line is gone yum is not going to update an rpm that is not already on the system. For instance yum will not update the DA rpm of MySQL-server with the Fedora rpm mysql-server. The two titles do not match.

But this is based on my limited 3 year experience with DA. We can talk in generalities all day. Do you have specific examples of problems you have experienced with yum updates?
 
You must log in or register to reply here.
Back
Top