Manual changes need with new version of LibXML2

[DirectAdmin Support]

I've updated custombuild with the new version of LibXML2:
Version: 2.7.8 (from 2.7.6)

and zlib:
Version 1.2.6 (from 1.2.3)

When updating these, you *must* update them as a pair.
zlib is *not* part of the "./build all d", hence you must manually run it first, eg:

cd /usr/local/directadmin/custombuild
./build update
./build zlib
./build all d

If the order is done wrong, these errors have been noted for a few of our test boxes:

Debian

/usr/local/lib/libxml2.so: undefined reference to `gzdirect@ZLIB_1.2.2.3'
collect2: ld returned 1 exit status

Other things we ran into, but also implemented other fixes in the build script to address them (you do still need to run ./build zlib first though)

CentOS 4

./.libs/libxml2.so: undefined reference to `gzdirect'
./.libs/libxml2.so: undefined reference to `gzopen64'

where we added "--with-zlib=/usr/local" to the configure line (it's already in the build script, so you shouldn't run into this particular error.)

FreeBSD 6

./configure.lineno: 14568: Syntax error: Bad substitution

which isn't specific to the zlib issue, but is a bug either with FreeBSD 6, or libxml2 on freebsd 6, related to this fix, which is already added into the build script.


Another related package is libxslt, who's version did not change (1.1.26), but you may want to recompile it anyway, as it is linked against libxml2. Note it's part of the "./build all d", so if you do the "all d" version, you shouldn't need to bother with this step, but if you're poking at the issue manually, it is noteworthy.

John

 

[ditto]

What does the d in ./build all d do?

Is every custombuild applications recompiled when doing ./build all d ?

I like to recompile applications in custombuild one by one, and not all automatically. What applications and in what order do I need to recompile in custombuild instead of ./build all d ?

 

[DirectAdmin Support]

The 3rd argument of ./build can be either 'y' 'n' or 'd'.
y: if anything is asked, y is answered (including, make failed, do you want to try again) which can get you into a loop.
n: same, except n is answered.
d: each type of question may have a specific best answer for most cases, so build uses the best answer.

For "make failed, do you want to try again?", it will be 'n'.
For "module is already installed. Would you like to build it again? ", it will be 'y'.

So, "./build php n" is the quick way to recompile php without the modules, while "./build php d" would be the best way to recompile it and all modules (mhash, crypt, etc.).
Using "./build php y" wouldn't be needed too much.. but ensures that y is in fact set for everything (even though 'd' should already be 'y' for modules)

John

 

[ditto]

Thank you for explanation. But you did not answer my other question. I like to recompile the only needed applications one by one, and not all automatically. What applications shouild I recompile, and in what order?

 

[DirectAdmin Support]

xml2 needs zlib, and apache uses zlib.. so most everything.. eg:

./build update
./build zlib
./build apache
./build libxml2
./build libxslt
./build freetype
./build curl
./build php n

./build suphp    #only if you use suphp, as ./build apache may empty /usr/local/lib

To get the modules, I did

ldd /usr/local/bin/php

and for each library it needs, I did an ldd on that library.
If library links to any of the new modules (libz or xml2), I added it to the above list.

John

 

[ditto]

Thank you, John! I will try to do this then:

./build update
./build zlib
./build apache
./build libxml2
./build libxslt
./build freetype
./build curl
./build php n
./build suphp

Edit: Is it needed to ./build pcre ? Because I have pcre version 8.21, but custombuild has dowgraded pcre to version 8.20, but I would not like to downgrade.
I will update in few hours after I know if there is any trouble.

Edit2: I have now upgraded the first server using the commands above. I have not seen any problems, so far it seems to work perfect. However I still wonder if pcre needs to be recompiled?

By the way, I am running CentOS 6.2 64bit.

 

[ditto]

I am now upgrading the second server doing the ./build .... on the list of applications in previous post. Only thing I notice is that when doing ./build php n I get many lines like this:

/usr/local/bin/php: /usr/local/lib/libxml2.so.2: no version information available (required by /usr/local/bin/php)

But everything is working without any trouble. So I think that warning is fine?

 

[ditto]

I have now upgraded three servers using the commands in above post, and there is no trouble at all! I also feel that webpages are loading even faster then before! I think the new zlib version might be the reason for even faster loading times! Thank you, John!

 

[nmb]

Got error after update which is similar to your CentOS4 error that said if I'm not follow steps but I did follow steps here

./build update
./build zlib
./build apache
./build libxml2
./build libxslt
./build freetype
./build curl
./build php n

Below is the error after run ./build php n

checking for mysql_close in -lmysqlclient... yes
checking for MySQL UNIX socket location... no
checking for MySQLi support... yes
checking whether to enable embedded MySQLi support... no
checking for mysql_set_server_option in -lmysqlclient... no
configure: error: wrong mysql library version or lib not found. Check config.log for more information.

*** There was an error while trying to configure php. Check the configure/suphp/configure.php5 file

My config.log shows :

configure:60979: checking for MySQL UNIX socket location
configure:61352: checking for MySQLi support
configure:61396: checking whether to enable embedded MySQLi support
configure:61547: checking for mysql_set_server_option in -lmysqlclient
configure:61566: gcc -o conftest -I/usr/local/include -g -O2 -fvisibility=hidden -Wl,-rpath,/usr/lib64 -L/usr/lib64 -L/usr/local/lib -Wl,-rpath,/usr/local/lib -L/usr/local/lib -Wl,-rpath,/usr/kerberos/lib64 -L/usr/kerberos/lib64 -lmysqlclient -lm -lrt -ldl conftest.c -lmysqlclient -lmysqlclient -lmcrypt -lltdl -liconv -lfreetype -lpng -lz -ljpeg -lcurl -lz -lpcre -lrt -lm -ldl -lnsl -lxml2 -lz -liconv -lm -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lssl -lcrypto -ldl -lz -lcurl -lxml2 -lz -liconv -lm -lssl -lcrypto -ldl -lz 1>&5
/usr/local/lib/libxml2.so: undefined reference to `gzdirect@ZLIB_1.2.2.3'
/usr/local/lib/libxml2.so: undefined reference to `gzopen64@ZLIB_1.2.3.3'
collect2: ld returned 1 exit status
configure: failed program was:
#line 61555 "configure"
#include "confdefs.h"
/* Override any gcc2 internal prototype to avoid an error. */
/* We use char because int might match the return type of a gcc2
builtin and then its argument prototype would still apply. */
char mysql_set_server_option();

int main() {
mysql_set_server_option()
; return 0; }

My server is running CentOS 5.7 64-bit, PHP 5.3, MySQL 5.5 and also Apache 2.2/2.4. (Try both Apache version with the same error.)

Below is /usr/local/lib directory

-rwxr-xr-x  1 root root     975 Mar  1 11:15 libxml2.la
lrwxrwxrwx  1 root root      16 Mar  1 11:15 libxml2.so -> libxml2.so.2.7.8
lrwxrwxrwx  1 root root      16 Mar  1 11:15 libxml2.so.2 -> libxml2.so.2.7.8
-rwxr-xr-x  1 root root 4151928 Jul 24  2011 libxml2.so.2.7.6
-rwxr-xr-x  1 root root 4159634 Mar  1 11:15 libxml2.so.2.7.8
-rw-r--r--  1 root root 1509124 Mar  1 11:16 libxslt.a
-rwxr-xr-x  1 root root    1004 Mar  1 11:16 libxslt.la
drwxr-xr-x  2 root root    4096 Jul 24  2011 libxslt-plugins
lrwxrwxrwx  1 root root      17 Mar  1 11:16 libxslt.so -> libxslt.so.1.1.26
lrwxrwxrwx  1 root root      17 Mar  1 11:16 libxslt.so.1 -> libxslt.so.1.1.26
-rwxr-xr-x  1 root root  822296 Mar  1 11:16 libxslt.so.1.1.26
-rw-r--r--  1 root root  125354 Mar  1 11:11 libz.a
lrwxrwxrwx  1 root root      13 Mar  1 11:11 libz.so -> libz.so.1.2.6
lrwxrwxrwx  1 root root      13 Mar  1 11:11 libz.so.1 -> libz.so.1.2.6
-rwxr-xr-x  1 root root   97932 Jul 24  2011 libz.so.1.2.3
-rwxr-xr-x  1 root root   96942 Mar  1 11:11 libz.so.1.2.6

 

[nmb]

Just tried

./build all d

and still got same error

 

[nmb]

Revert back to libxml2 2.7.6 and zlib 1.2.3. No more error....

 

[DirectAdmin Support]

Hello,

I just got off of a CentOS 6, 64-bit box and it was refusing to use the new version of xml2 and zlib, with the same error as post #9.
It looks like when php is linking to libxm2, it's not using the correct zlib library, even though we've got --with-zlib-dir=/usr/local/lib set (if that even applies for the xml2 linking.. not sure).
As nmb mentioned, reverting to the old versions was the only thing I could do.
To do this, I ran:

cd /usr/local/directadmin/custombuild
perl -pi -e 's/libxml2:2.7.8:.*/libxml2:2.7.6:/' versions.txt
perl -pi -e 's/zlib:1.2.6:.*/zlib:1.2.3:/' versions.txt
rm -f /usr/local/lib/libz.so*
rm -f /usr/local/lib/libxml2.so*
ldconfig
./build zlib
./build libxml2
./build php n

which did work.. but is a rather dangerous bit of code (removal of libz is risky).
Note that there is going to be another copy in /lib64/libz*, hence the running of "ldconfig" right after removal, so binaries link to the old location.
I did regular checks with:

ldd /usr/sbin/sshd | grep libz

to ensure it was pointing to something that existed.

I'll update this thread if I find more info about it.

John

 

[DirectAdmin Support]

For now, until a better solution is found, I've reverted xml2 back to 2.7.6 (from 2.7.8) and zlib to 1.2.3 (from 1.2.6) in the versions.txt.

John

 

[ditto]

Hi John, I have already upgrade all 3 of my servers to zlib 1.2.6 and libxml 2.7.8. I have no problems at all, it works perfect. I am using CentOS 6.2 64bit

I do not want to downgrade. But you say you have downgraded in custombuild, will that give me problems? I don't want the 6 year old zlib version! That is too old!

I did this:

./build update
./build zlib
./build apache
./build libxml2
./build libxslt
./build freetype
./build curl
./build php n
./build suphp

Again, I do not have any problems with the new versions. Please fix the problem and add new versions to custombuild. I don't want to downgrade! It is not a option to downgrade for me!

Edit: I am using Apache 2.2.22 (not the new 2.4.1). Maybe all the problems other people report, is because of the new Apache 2.4.1?

Edit2: Also I am using PHP 5.3.10 with suPHP.

 

[ditto]

Maybe the people reporting trouble was all doing brand new installs? I was not, I was upgrading existing servers. And I did not have any problems! I upgraded one xen vps, and two dedicated servers. No problems.

If you like, John, I can give you root access to one of my servers, if it will help to look at my settings?

I will be very displeased if you do not fix the problem and add back the new versions to custombuild. I will not continue to use DirectAdmin if I have to wait additional years for you to upgrade zlib wich already is more then 6 years old. I will not downgrade.

 

[ditto]

Got error after update which is similar to your CentOS4 error that said if I'm not follow steps but I did follow steps here

*** There was an error while trying to configure php. Check the configure/suphp/configure.php5 file

Maybe the reason for the error is something with suphp? Because suPHP was not compatible with Apache 2.4.1, and maybe smtalk patched suPHP in Custombuild to work with new Apache version, and maybe that patch is the reason for this problem?

I upgraded zlib and libxml in 3 servers without this problem, but I upgraded it before smtalk patched suPHP I think.

Did smtalk patch and change suPHP? If so, that is your problem then.

John, please check that, and revert to suPHP without smtalk patch for new Apache!

 

[nmb]

Well, ditto, I have my server running for ... a year, I think after clean installation because of some problem. So, it's not new install... And yes, I tried both Apache 2.2 and 2.4.1 with the same problem. So, I don't think it's because of suphp patch that smtalk did it. (But well, I got problem with mod_fastcgi can't compiled which it does before smtalk patched suphp. However, I don't know if that is the problem or not. Will try it again later.)

Revert back to old zlib / libxml2, I can run Apache 2.4.1 without any problem. Also, you might need to take a look at http://www.directadmin.com/forum/showthread.php?t=43011&p=218116 Not only me who got that problem. Even with different OS version, we still got same error. (Mine is CentOS 5.7 and their is CentOS 6.x)

Oh.. and yes, I always like to try new version even I know I shouldn't because it will affect all the websites in my server. Still, I tried it and hoped that it will work but it didn't. I also tried this because you said you have no problem with 3 servers. Still, I'm not lucky as yours. After many hours of trying, I just gave up though. So, a newer version is great but at least, I want it to work flawlessly.

 

[DirectAdmin Support]

Hello,

1) If you've already updated, you don't need to downgrade if you don't want to. Even if the old xml2 is compiled, ldconfig will still point to your newer version, so the new versions will be left there and used unless physically removed and ldconfig run.

2) I've checked all of our latest build boxes (except FreeBSD 9 64, as we don't have one). They all still use zlib 1.2.3, so I don't think we're alone in staying with this version.
The reason is that zlib is a low level library and many services are linked to it, so if zlib is updated, in theory, so should all services that link to it "to be safe".

The way I checked the version is by physically checking the packaged libz.so library vs ours with the readelf -Ws command, eg:CentOS 6 64-bit

[root@es6-64 ~]# cat /etc/redhat-release
CentOS Linux release 6.0 (Final)
[root@es6-64 ~]# uname -m
x86_64
[root@es6-64 ~]# readelf -Ws /lib64/libz.so.1 | grep gzopen64
[root@es6-64 ~]# readelf -Ws /usr/local/lib/libz.so.1 | grep gzopen64
    64: 000000000000e250    13 FUNC    GLOBAL DEFAULT   12 gzopen64@@ZLIB_1.2.3.3
   137: 000000000000e250    13 FUNC    GLOBAL DEFAULT   12 gzopen64
[root@es6-64 ~]#

where both gzopen and gzopen64 exists in 1.2.6, and 1.2.3 just has gzopen.
So our latest boxes are not packaged with the newer zlib. Even with a yum update of zlib, it still provides 1.2.3. (I actually found that Debain 6 64 has version 1.2.3.4, which does have gzopen64, but that OS doesn't have issues, so doesn't matter)

I have a feeling that newer OS's (perhaps FreeBSD 9) will package the newer libz (and xml2) as all services will be fresh and able to be compiled against it.

What I can do, is simply add a custombuild options.conf setting like "new_zlib=yes", where the default is no.
If you want the new zlib, set it to yes, and the versions.txt will have extra entries for the newer version of zlib and xml2.

However, if you OS is still not packaged with the newer version of zlib, there may be a good reason, so having the new version may not always be in your best interest.

We'll be continuing to monitor and assess the issue, it's not yet "closed", but the new_zlib=yes|no option may be the simplest route.

John

 

[ditto]

[...] The way I checked the version is by physically checking the packaged libz.so library vs ours with the readelf -Ws[..]

I did your commands on one of my servers (a Xen vps), and here it is (same result I think):

[root@dns ~]# cat /etc/redhat-release
CentOS release 6.2 (Final)
[root@dns ~]# uname -m
x86_64
[root@dns ~]# readelf -Ws /lib64/libz.so.1 | grep gzopen64
[root@dns ~]# readelf -Ws /usr/local/lib/libz.so.1 | grep gzopen64
    64: 000000000000e250    13 FUNC    GLOBAL DEFAULT   12 gzopen64@@ZLIB_1.2.3.3
   137: 000000000000e250    13 FUNC    GLOBAL DEFAULT   12 gzopen64
[root@dns ~]#

...I have a feeling that newer OS's (perhaps FreeBSD 9) will package the newer libz (and xml2) as all services will be fresh and able to be compiled against it.

What I can do, is simply add a custombuild options.conf setting like "new_zlib=yes", where the default is no.
If you want the new zlib, set it to yes, and the versions.txt will have extra entries for the newer version of zlib and xml2.

Yes, please do this! As said previous, I have upgraded all three of my servers, and I don't yet see any trouble on them. Also I would be very scary to downgrade, I would not be able to downgrade without very detailed instructions.

I was so happy when I upgraded and it worked perfect! About two years ago, I investigated alot trying to find way to upgrade zlib without Custombuild. But I did not do it. So when you released new versions, and I upgraded without any problems, I was very happy.

Also a phpinfo page on all three servers, shows that the new zlib version is in php, in both curl and its own listing. phpinfo page also display the new libxml version. And I don't have any errors or trouble with anything.

...However, if you OS is still not packaged with the newer version of zlib, there may be a good reason, so having the new version may not always be in your best interest.

We'll be continuing to monitor and assess the issue, it's not yet "closed", but the new_zlib=yes|no option may be the simplest route.

I think Redhat/CentOS always is very slow to add new versions, but instead do alot of patching. It does not seem to be that there is any good valid reason for it, its just the way it is.

I think zlib 1.2.3 is too old, one has to upgrade sometime. 2 and 3 years is fine, but 6 years, that is alot in todays world.

Edit: I notice you have CentOS 6.0, but I have CentOS 6.2. Maybe the new zlib and libxml only work in 6.2? ...

 

[DirectAdmin Support]

I notice you have CentOS 6.0, but I have CentOS 6.2. Maybe the new zlib and libxml only work in 6.2? ...

as mentioned, it works fine on our 6.0 64-bit box.
We were not able to figure out why it doesn't work on some others.

In any case, I've added the option to custombuild for the options.conf:

new_zlib=no

which will use the latest zlib and xml2 (set in the versions.txt as libxml2-current and zlib-current)
Update custombuild and set this to "yes" if you want the new versions.
It's in files1.directadmin.com right now, and will be in the other mirrors within 24 hours.

John