Unidentified process running constantly

srqwebguy

Verified User
Joined
Apr 22, 2007
Messages
77
For the last few days we have had a process running constantly that is consuming anywhere from 52% to 80% of CPU resources. We cannot track down what is doing this, and even though all sites except one low traffic site have been moved off it we still see this issue. The log files really are not telling us anything. The process is /usr/bin/top -c -b -n 1

From Process Monitor:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
15230 root 22 0 2184 936 700 R 62 0.0 0:01.49 /usr/bin/top -c -b -n 1

Any idea what this process may be doing? More importantly, how do we kill it?
 

zEitEr

Super Moderator
Joined
Apr 11, 2005
Messages
14,255
Location
GMT +7.00
Hello,

DESCRIPTION
The top program provides a dynamic real-time view of a running system. It can display system summary information as well as a list of processes or threads currently being managed by the Linux kernel. The types of system summary information shown and the types, order and size of information displayed for processes are all user configurable and that configuration can be made persistent across restarts.

The program provides a limited interactive interface for process manipulation as well as a much more extensive interface for personal configuration -- encompassing every aspect of its operation. And while top is referred to throughout this document, you are free to name the program anything you wish. That new name, possibly an alias, will then be reflected on top's display and used when reading and writing a configuration file.



Read more with

Code:
man top
That might be directadmin which checks and collects data from top.
 

srqwebguy

Verified User
Joined
Apr 22, 2007
Messages
77
Thnaks. We've used top both within the process monitor and SSH. We've never seen it continually consume so much processor capacity, and slow the machine to a crawl (many items within DA are now timing out). It seems to be continually running. How do we force top to stop?
 

r3chn3r

Verified User
Joined
Jan 13, 2013
Messages
137
You could kill it, but it is likely that you're just seeing the process you are using to view processes.

Are you sure that you're not getting ddos attack to a wp-login.php? Seems to be happening everywhere.
 

srqwebguy

Verified User
Joined
Apr 22, 2007
Messages
77
True, as I had that on another machine recently. This machine has no Wordpress installed.

I know top uses some resources, but on other machines it is like 3 or 4%.
 
Last edited:

srqwebguy

Verified User
Joined
Apr 22, 2007
Messages
77
Another strange finding - I went to look at the access logs again based on what you said, and the httpd/access_log is empty. 0 bytes.
 

srqwebguy

Verified User
Joined
Apr 22, 2007
Messages
77
Right now it reads as follows. One other problem, as evidenced here, is it is still trying to complete a backup that was started about 14 hours ago. It normally backs up nightly to a remote server without an issue, but the last 4 days the process has started and then hangs up. I discovered and eliminated 3 processes yesterday, each trying to back up the admin account. Each had been started one day apart. One was 25% complete and the other 2 were 10% complete.

ffssl-screenshot.jpg

Edited to attach a screenshot. Easier to read than below.

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27250 root 18 0 9352 2784 2372 R 95 0.1 1:19.14 /usr/local/directadmin/dataskq
27312 root 23 0 2184 940 700 R 59 0.0 0:01.64 /usr/bin/top -c -b -n 1
27292 root 18 0 9680 1908 1148 R 36 0.0 0:03.62 /usr/local/directadmin/directadmin d
27050 diradmin 34 19 1944 852 728 S 6 0.0 0:13.15 /usr/bin/ncftpput -f /home/tmp/admin/user.admin.flashform.tar.gz.cfg -V -t 120 -P -m /Server_3 /home/tmp/admin/user.admin.flashform.tar.gz
220 root 16 0 0 0 0 S 4 0.0 1:20.24 [pdflush]
 
Last edited:

zEitEr

Super Moderator
Joined
Apr 11, 2005
Messages
14,255
Location
GMT +7.00
The 5 very top lines of top output as how I see it in my case are as following:

Code:
top - 02:03:12 up 28 days,  2:32,  1 user,  load average: 0.08, 0.03, 0.00
Tasks:  93 total,   1 running,  92 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.3%us,  0.7%sy,  0.0%ni, 99.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    502204k total,   427300k used,    74904k free,    15800k buffers
Swap:  2096124k total,        0k used,  2096124k free,   259144k cached
I'm curious to see yours.
 

srqwebguy

Verified User
Joined
Apr 22, 2007
Messages
77
Sorry. Misunderstood.

Code:
top - 14:05:36 up 1 day, 3:34, 1 user, load average: 0.83, 1.11, 1.23
Tasks: 170 total, 1 running, 169 sleeping, 0 stopped, 0 zombie
Cpu(s): 2.6%us, 1.3%sy, 13.6%ni, 81.3%id, 0.3%wa, 0.0%hi, 0.9%si, 0.0%st
Mem: 4072004k total, 3913828k used, 158176k free, 63352k buffers
Swap: 2040244k total, 156k used, 2040088k free, 3646488k cached
 

zEitEr

Super Moderator
Joined
Apr 11, 2005
Messages
14,255
Location
GMT +7.00
Do you see this during higher load? Or during normal operation?
I'd like to see data during higher load.
 

srqwebguy

Verified User
Joined
Apr 22, 2007
Messages
77
Right now we've moved almost all functions off this machine, due to its lagging response times. No publicly active websites. This server is currently only managing an administrative area and related database for another website on a different server. That system does feed some index page content from MySQL, but only a few hundred calls a day. It has very little to do now.
 

zEitEr

Super Moderator
Joined
Apr 11, 2005
Messages
14,255
Location
GMT +7.00
Directadmin runs top the way it's specified in configs:

Code:
load_top_string=/usr/bin/top -c -b -n 1
Related: http://www.directadmin.com/features.php?id=1452

But it's triggered when Directadmin detects higher load on your server. It uses top to create a report for administrator of the server. And here is a template: /usr/local/directadmin/data/templates/load_check_message.txt

So something else was generating higher load on your server and now it's hard to say what was it exactly.
 

srqwebguy

Verified User
Joined
Apr 22, 2007
Messages
77
That's helpful to know. Thank you. We are continuing to investigate. The fact that some functions, like apache logs, are not working makes it difficult to track down. We have other machines and have migrated almost all off this. We will drop the machine if we can't fix it, but I like to try and learn every time we come across a new issue.

Your input has been very much appreciated.
 
Top