Server Hacked

R

rohit

Verified User
Joined
Mar 20, 2007
Messages
198
Location
Melbourne
We had a server (centos 5.2) with DA (version was the one before 1.36.0) with over 100 sites on them.

- The server was behind a Hardware Based firewall (WatchGuard) with the following ports open to public (80, 110, 143, 25, 21, 53, 443, 2222)
- SSH port 6022 was only open to our IP 202.62.33.2
- The server was running apache 1.3, with php 4.4.9 and mysql 4.
- There was also a software based firewall (APF) configured with BFD. APF was configured and was only allowing the above mentioned ports only including 6022.
- root & admin user ssh was disabled and we created a local user that was allowed to ssh
- php disable_functions=”proc_get_status,proc_nice,proc_open,proc_terminate,proc_close,dl,phpinfo,system,posix_kill,popen,exec,passthru,apache_note,apache_setenv,openlog,closelog,syslog,pcntl_exec,pclose,ini_restore,escapeshellcmd,escapeshellarg,define_syslog_variables”
- mod_evasive, mod_security were installed with apache
- php open_basedir was ON. Cant remember the settings for SAFE mode though

The hacker was able to delete /var/log folder and he also deformed the home page of most / all of the website(s).

When I logged into the server, I was able to see some perl scripts running(perl pull.pl).

Now my question is how in such a closed environment can this thing happen? What other measures can we take to make sure that this doesn’t happen again?

I am suspecting that the root cause is the webserver, but how was the hacker able to upload and run a script as a root user?

Any help would be much appreciated.

PS: i have attached the hacked example page
 

Attachments

  • hacked.JPG
    hacked.JPG
    43.2 KB · Views: 312
I agree that your server seems fairly secure. Because the /var/log/ subdirectory was deleted you may never know exactly how the hacker got in.

You can start, though with the ownership of the example page you posted.

Is it owned by apache? Is it owned by each individual site user? Is it owned by root?

If the former, then it could be a simple PHP hack, but then the question of how the user escalated to root remains a question.
You definitely need to rebuild your server. If you haven't already wiped the drive, I'd recommend rebuilding on a new drive and saving the old one.

Focus on any CMS systems hosted on your server as possible attack vectors.

Look up versions of every CMS used on your box and do not restore any CMS-based site if the current version has a history of being subject to being hacked.

Also focus on recently added users; especially any with shell access, as it would be easiest for shell users to attack your server. But probably more likely through a hole in a CMS running php.

Jeff
 
Probably an insecure script running on one of the sites. Its the most popular way of entry. Also, the OP does not talk about the kernel. Was it up to date or what?
 
I was able to track down what happened and found out the culprit website. Here is a run down

1. We host a clients website which has an upload feature (PHP based).
2. Hacker was able to crack the admin / staff login details which let him upload a perl script (back.pl), which happens to a WEB SHELL script
3. Hacker then also created .htaccess file in the same folder which had Options +ExecCGI or something in it.
4. Hacker I think then used one of the exploits of the kernel to gain root access.
5. Hacker was also able to upload mass.pl to the tmp folder, which defaces all the index(*) files on the server and then deletes all the logs folders and the history files.

how can we prevent (WEB SHELLs), which can be written in PHP, Perl, C etc etc? I have already followed the security tip #14 on http://help.directadmin.com/item.php?id=247. Is this enough? I read somewhere which mentioned about putting some rules for outgoing packets filtering. How can this be done? We are using apf for our firewalling.

How can someone upload a file which as apache:apache permissions be able to run a perl script when we didn’t open any Perl / CGI for that account? is it because of +ExecCGI?

PS: We have upgraded all our php and apache versions.

I found this URL , which explains about all the vulnerabilities on different linux kernels

http://r00tsecurity.org/forums/topic/4819-begginers-rooting-guide/

How oftern should we be upgrading the kernel as I know sometimes upgrading the kernels stops the server all together.

I am still trying to figure out all the steps that one should follow for a proper server security and also the maintenance activities that a sys admin should follow to keep the server up-to-date. Any pointers would be greatly appreciated.
 
hi rohit,

on this 100+ website that you got, do you have 1 or more clients that are using osCommerce?
If yes, take a look on them, because for 70% of chance, is because of that.

Another thing, linux distributions MUST be upgraded each week, at least to check if there are upgrades regarding all the package...
In this days, for so and so 1 month, connect on your servers and check periodically the syslog and the auth.log and block on iptables the ip that are trying to DNS flood you or trying to hack the ftp password, i'm telling this because in the start of august i received a similar attack, but i'm sure that it was because of php hacking using oscommerce, and than, after that i removed partition and reinstalled everything, i started to receive on syslog and on auth.log a lot of trial to DNS flood and ftp hacking...


Let me know, if you need other suggestion, read also my thread and the answer that the people gave to me :)
 
rohit said:
I was able to track down what happened and found out the culprit website. Here is a run down

1. We host a clients website which has an upload feature (PHP based).
2. Hacker was able to crack the admin / staff login details which let him upload a perl script (back.pl), which happens to a WEB SHELL script
3. Hacker then also created .htaccess file in the same folder which had Options +ExecCGI or something in it.
4. Hacker I think then used one of the exploits of the kernel to gain root access.
5. Hacker was also able to upload mass.pl to the tmp folder, which defaces all the index(*) files on the server and then deletes all the logs folders and the history files.

how can we prevent (WEB SHELLs), which can be written in PHP, Perl, C etc etc? I have already followed the security tip #14 on http://help.directadmin.com/item.php?id=247. Is this enough? I read somewhere which mentioned about putting some rules for outgoing packets filtering. How can this be done? We are using apf for our firewalling.

How can someone upload a file which as apache:apache permissions be able to run a perl script when we didn’t open any Perl / CGI for that account? is it because of +ExecCGI?

PS: We have upgraded all our php and apache versions.

I found this URL , which explains about all the vulnerabilities on different linux kernels

http://r00tsecurity.org/forums/topic/4819-begginers-rooting-guide/

How oftern should we be upgrading the kernel as I know sometimes upgrading the kernels stops the server all together.

I am still trying to figure out all the steps that one should follow for a proper server security and also the maintenance activities that a sys admin should follow to keep the server up-to-date. Any pointers would be greatly appreciated.
Click to expand...

this hacker used the same steps used to hack mine. Uploading the .pl file and have the .htaccess setted on execcgi, he can run the script on the server, all the session will be setted on /tmp/ directory, and if he will got the root with some exploit, for sure, in the end, was able to defaces all the index.* / home.* files (like the turkish group did to me).
 
rohit said:
5. Hacker was also able to upload mass.pl to the tmp folder, which defaces all the index(*) files on the server and then deletes all the logs folders and the history files.
Click to expand...

As for files in tmp: make tmp noexecutable.

Step by step at http://www.directadmin.com/forum/showthread.php?t=29807#13

You could also considering using CSF, which is what that thread is about. It includes lfd instead of bfd and it has a security checking page which among other things checks your /tmp on noexec. It also has a nice integration with DA.
You would need to uninstall APF, I personally can't say which one is better but what I generally read is that CSF is preferred.
 
You must log in or register to reply here.
Back
Top