Server Nearly Hacked

C

charliecreed

Verified User
Joined
Feb 26, 2006
Messages
56
Hello,

Basiclay today I woke up as usal then see emails from my ISP showing my box send outgoing data to random IP's to try and infect a box.

I logged in and see thousands of process's of sshd running by apache username. loads were arround 5.00 Luckly It didnt kill the box due to the amount of power the box has itself.

I killed the sshd then booted it back up through webadmin, logged back in and load was down, The first thing I did was check the logs.

Here is what I found in /var/log/httpd/error_log:

[Wed Apr 19 00:11:04 2006] [notice] Apache/1.3.34 (Unix) mod_ssl/2.8.25 OpenSSL/0.9.7g PHP/4.4.2 FrontPage/5.0.2.2510 configured -- resuming normal operations
[Wed Apr 19 00:11:04 2006] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Wed Apr 19 00:11:04 2006] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Wed Apr 19 01:43:17 2006] [error] [client 127.0.0.1] File does not exist: /var/www/html/r57shell/version.php
sh: sysctl: command not found
sh: sysctl: command not found
sh: sysctl: command not found
sh: sysctl: command not found
loads of sysctl command not found. not going to paste all of them.
--01:44:13-- http://www.el-z.org/privat/ownz
=> `ownz'
Resolving www.el-z.org... 66.6.63.32
Connecting to www.el-z.org|66.6.63.32|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 492,947 (481K) [text/plain]

0K .......... .......... .......... .......... .......... 10% 115.94 KB/s
50K .......... .......... .......... .......... .......... 20% 231.78 KB/s
100K .......... .......... .......... .......... .......... 31% 437.02 KB/s
150K .......... .......... .......... .......... .......... 41% 237.70 KB/s
200K .......... .......... .......... .......... .......... 51% 446.75 KB/s
250K .......... .......... .......... .......... .......... 62% 230.71 KB/s
300K .......... .......... .......... .......... .......... 72% 425.81 KB/s
350K .......... .......... .......... .......... .......... 83% 229.53 KB/s
400K .......... .......... .......... .......... .......... 93% 237.97 KB/s
450K .......... .......... .......... . 100% 295.69 KB/s

01:44:16 (246.62 KB/s) - `ownz' saved [492947/492947]
sh: sysctl: command not found
--01:45:13-- http://www.geocities.com/ijookeren/bind.txt
=> `bind.txt'
Resolving www.geocities.com... 66.218.77.68
Connecting to www.geocities.com|66.218.77.68|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6,439 (6.3K) [text/plain]

0K ...... 100% 43.52 KB/s

01:45:14 (43.52 KB/s) - `bind.txt' saved [6439/6439]
[Wed Apr 19 05:53:38 2006] [error] [client 127.0.0.1] File does not exist: /var/www/html/r57shell/version.php
( lots of that every so often )

That's all I have been able to digout so far. I downloaded the files which he downloaded. The ownz file has to be complied with gcc so I didnt look at it however the bind.txt shows what is going on

As it says in that file type ps -A to see if the process is up and there was one called ownz, I quickly killed that.

Then I did a few kernel updates to the box, then did some advanced iptable rules just incased he tried this again.

Iptables will fix it for the time being, but still I want to know how he did this and how to stop it from happening again.

if you have any ideas to check out different logs please reply and I'll paste you whats in them.

Also the system is using Ubuntu but basiclay Debian.

Thanks,
Charlie
 
Last edited:
I have seen the same hack (based on the r57shell you reported) a couple of weeks ago. In my case they had exploited a vulnerability in phpMyAdmin (after upgrading phpMyAdmin that hole was plugged).

After that frightning experience I did a lot of reading and found lots of useful tips on how to secure your server. One of them was particularly useful, and very simple as well. I discovered in the httpd logfiles that almost always these hackers are able somehow to incorporate a "wget" command into the url. The tip suggested making the wget command owned by root:root and setting access-rights to 700. That way the apache process won't be able to execute wget and the hacker won't be able to download anything to the server (at least not using wget). I'm pretty sure there are other techniques as well that hackers can use to break in, but at least this will make it a little harder for them.

I also wanted to have some warning system that would alert me of strange activities on my server. Of course there's snort, but it monitors network traffic. I wanted something to check my filesystem as well. So I started searching and I in a short while I had a list of software that did just that (tripwire, aide, fcheck, samhain, osiris etc.). After spending hours and hours reading about these alternatives I finally decided to give samhain a try (it seems to be one of the most mature IDS's of the ones I found). But before I dare to install it I first ordered a book about Osiris/Samhain (ISBN: 1-597490-18-0). I'll first read that book before installing anything because I cannot afford to "break" anything on my server :)
Anyway, I'll post my experiences with Samhain shortly (I started a thread about it in the third-party-software forum last week, asking for experiences from others. I'll add to that thread telling how it went).

Good luck with your server!
 
Last edited:
A few other things that might help, is making sure that /tmp is mounted with noexec. If you do that, make sure you specifiy an alternate temp directory for logrotate, or else that will break. Also, make programs like lynx, links, elinks, ... 700. gcc, g++, ... should also be 700, noone but the admin should have to compile anything.
 
Ok, I have done all this thanks for the tips.

Is there anyway I can stop things from being run by apache like he was doing or?
 
charliecreed said:
Is there anyway I can stop things from being run by apache like he was doing?
Click to expand...
Sure, that's what you did when you changed ownership to root:root and access-rights to 700. That will prevent apache from running those commands.

There are other candidates as well for this modification besides wget and the commands that toml mentioned (curl for example, and rsync and a few others) but I've never seen those being used in the logfiles.
Basically any command that can transfer files over the network could potentially be misused so you may want to modify those as well.

Also regularly check for updates for any scripts installed on your server, either by you or any of your customers. According to my logfiles to most 'popular' scripts they are looking for are phpMyAdmin, Mambo, Joomla, Coppermine, Webcalendar, Awstats and Horde. And make sure you enable open_basedir (or even php safemode), otherwise a hacker can modify files in any homedir, not just the one who's site was hacked.
 
he used wget to download ownz.

generally to prevent this a few things that would have worked is.

disable world access to tools that are used to download files to server such as.

wget
lynx
fetch
scp
ftp

also to help prevent downloaded tools been executed restrist the /tmp mount to noexec and nosuid, this can be traversed fairly easy but should stop some automated scripts at least.

open_basedir will help stop dir traversal.

mod_security again help stop someone using wget etc. through apache.
 
I have done everything everyone has suggested and limited outgoing connections through iptables.

So far so good :)
 
Use mod_security to close the ftp,wget,lynx,fetch ex. holes

mod_security also can be used to close SQL Injection attempts and much more powerful tool, that with the right config can help prevent alot of hacking attempts
 
You must log in or register to reply here.
Back
Top