Help, server hacked...

B

Brian S

Verified User
Joined
Oct 27, 2003
Messages
57
So I wake up today to find out my server's been hacked.

I think he might have come through a PHP script or BIND.

I load up DirectAdmin CP login and find this:
-----------------
hacked by Pyske

uid=0(root) gid=0(root) groups=1024(tasmahal)

Linux [myhostname].com 2.6.9-78.0.13.EL #1 Wed Jan 14 16:00:26 EST 2009 i686 i686 i386 GNU/Linux

Alayina gider..

root:$1$mlFLlYaa$XvnpHDpzSYPVXBu.ig/Z20:13315:0:99999:7::: bin:*:13298:0:99999:7
------------------

It looks like all of the log directories have been deleted, and possible some library files.

I re-created the log directory and rebuild apache. It starts and the exact same message appears on all of the domains.

In /tmp/ I find this:
-------------------
back
bds
cmdtemp
-------------------

It looks the hack came through the user tasmahal, which has an old Joomla 1.x script on it according to client.

I also don't have a drive backup because it was disabled becuase it was bringing the server down every morning. Do you have any suggestions?

Thank you,

Brian
 
Lovely, looks like every single index* file on the server has been overwritten with this message...

I guess suPHP didn't do its job.
 
In order to do that much damage he would have to have root access. That did not happen from some php exploit alone.
 
That's what I was wondering. So basically the entire server has been compromised? That bds file looks like a program that was a BIND exploit.

I'm kinda up a creek without a current backup. I can't even access the old backups because I think the second (backup) drive is erroring out.

Ahhhhh, crappy day...
 
Sorry dude.

Some people are going to disagree with me here. This is what I would do if you have no backups at all.

I would run the admin/backups now so you have something to build from. Then reinstall the OS and everything and then run the restore.

If a rootkit was installed it is unlikely that the rootkit will be in the backups since that is mostly user data. Your users will have to fix their index files but hopefully that is all they will have to fix.

If a user script was somewhat responsible for this then it is still there and you will need to find it. How I don't know.

Tillo is our security expert here and he will have more advice for you. Try sending him a PM and maybe it will send him an email.
 
Thanks, I was looking at moving over to a cloud computing platform and I guess this will be my nudge. One problem is that this stupid hacking message has infected the DA control panel as well. I'm trying a command line update to see if that will fix it, but otherwise I'll have to reinstall DA before I can even backup data.
 
You do not need the web gui front end. Just write the command to the task.queue directly:

Code: 
echo "action=backup&local_path=/home/admin/admin_backups&owner=admin&type=admin&value=multiple&when=now&where=local&who=all" >> /usr/local/directadmin/data/task.queue

/usr/local/directadmin/dataskq &
 
Yep, I got DA working again, thanks. Currently migrating user data over to a new server. I'm trying out LiquidWeb's StormCloud hosting. I can automate backups, create server images, and scale servers up and down, all via their panel. Very simple and reasonable. Hopefully it will be reliable. Their rep seems to be good. I'm tired of dealing with hardware. :p

Thanks,

Brian
 
Storm Cloud looks interesting. I'm on hold with them on the phone now. Please keep us posted as to your experience.

Jeff
 
I've tried out GrGrid and Amazon EC2 and researched some others like SliceHost, VPSHive, and the Rackspace offerings. They all have some limitations. GoGrid, for example, limits the amount of storage space you get with each server, so you'd have to use their cloud storage, which presents file ownership and file locking issues because it's like NFS. Amazon still limits you to one IP per cloud server. The others didn't offer the right combo of resources (RAM, storage, CPU), or were too expensive to be practical for me.

I wish StormCloud had a control panel-based load balancer and cloud storage like GoGrid, but the fact that you can automate backups via the CP (think security in case of, ahem, being hacked), can scale a server up or down in minutes, or create a snapshot of a running server and instantly clone it--user data and all--are all strong features. And it's relatively reasonably priced. I hope it works as advertised. I don't have the server under a load yet, but otherwise, so far, so good.

Here's my chat transcript with LW:
Paul: Greetings, my name is Paul. Welcome to Storm On Demand! How may I assist you today?
you: Hi. If I create, say, a 4GB cloud, can I back it up, and then redeploy it as a 8GB cloud without losing any of the data? Basically, are the clouds scalable?
Paul: Absolutely!
Paul: That is one of main features of Storm
you: So I am using the cloud for webhosting, it's no problem to upgrade the cloud without losing any data? This is a major limitation of GoGrid.
Paul: Exactly
you: Great. About how long does it take to make the backup? An example would be a cloud with 200GB of data.
Paul: I am not exactly sure, but they should be faster than a normal rsync
you: So perhaps a couple hours? If im in a bind and a server is getting hit with a ton of traffic, I just wouldn't want it to be inaccessable to client updates for like 12 hours while I'm backing up and redeploying it to a bigger cloud server.
Paul: Exactly, since it backs up data in a snapshot format vs. flie by file
Paul: its going to be faster that way
you: I see this question was answered on your website: Simply select resize, select your required server size and details and Storm copies and deploys your server in its identical configuration on your new server. A process that used to take days or weeks can now be done in a few mouse clicks!
you: How is CPU allocated? Do I received a certain guaranteed minimum per GB of RAM? Is there a real-world CPU equivalent that I can relate to?
Paul: Yes, with the 2GB and 4GB you get one and two vcpu allocated
you: What would be a real-world equivelent I can relate a vcpu to? e.g. Is a vcpu like a p4 3Ghz?
Paul: The hardware varies, but yes it would be similar to that type of cpu in performance
you: okay. And you have base CentOS images with no CP, so I can bring my own DirectAdmin CP and install it?
you: CP=control panel
Paul: Yes we do
Paul: We only manage the cpanel instances though
you: Okay thats no problem. Last question, I think :-) Let's say I sign up for this now, and try it out for two or three days, but find it's not working for my needs. Is it possible to get a refund?
Paul: You only pay for what you use
Paul: like a utility
you: So if I use a cloud for five days, and close my account, I'm only charged a pro-rated amount?
Paul: right
you: Perfect. I lied, last question. Is support available 24x7 for issues like needing more IPs or a functional issue with a cloud?
Paul: Yes
Paul: You can add more yourself in the dashboard
Paul: up to 8 per instance
you: Yeah, I was looking at that. I just wanted to know if I needed a couple extra IPs--over the 8-- for a client, I could email or call and have them allocated within a couple hours.
Paul: Yes, we can do that
you: Cool. Well, my questions have been answered. Seems yall have a good rep, so I'll give this a go. Happy to be supporting the MI economy, too. ;-)
Paul: Great! Glad to hear it, welcome aboard!
you: Thanks. Have a good one.
Paul: You as well, take care
Click to expand...

After joining, the couple tickets I submitted on a Sunday were answered within 15 minutes. A good sign.

Brian
 
A very good sign. Can you give us an idea of what the tickets were about?

Any real issues?

Jeff
 
Simple stuff. Asked general questions about how backups worked, etc. I requested more IPs yesterday at 4 a.m. Eastern and got them in an hour.

Brian
 
Just an update, things have been running smoothly at LW's Storm on Demand service. Support continues to be good. They even logged into the server and restarted the SonarPush service the provides the reporting to the StormCP when they noticed it was down. I was able to resize the server to the next highest server. Only hiccup was, when I downsized back to the 4GB server, it hours to prepare and sync data, and then it was down for at least an hour while it did a final data sync and brought the re-sized server back up. I'm guessing it might have been a or load issue, becuase normally it take about 30 mins or less to do a full resize from the time I click the button to when the new server is up.
 
Thanks for the update. Resizing is always going to take some time on most clouds; did you ask them why it took so long?

Do they require the ability to log in? I'm not sure I'd want to give that to them.

Jeff
 
downsizing is always longer as the calculations and block size etc are more complex. Typically you would always be upsizing not not downsizing.
 
jlasman said:
Thanks for the update. Resizing is always going to take some time on most clouds; did you ask them why it took so long?

Do they require the ability to log in? I'm not sure I'd want to give that to them.
Click to expand...

I didn't ask. I've never provided a password and it's been changed through the shell, so I imagine they have some facility to login.

Thanks for the info Wisehosting.
 
You must log in or register to reply here.
Back
Top