$#!tstemd cgroups CPU, RAM, IO, (TASKs?) made easy

youds said:
Hmpff. Am stuck! It must be something I'm doing wrong. I've done it 4/5 times now on fresh installs to no avail.

Here is what I'm doing

Code: 
vi /etc/default/grub
grub2-mkconfig -o /boot/grub2/grub.cfg
shutdown -r now

And then...
Code: 
useradd test1
id -u test1
systemctl set-property user-1000.slice CPUQuota=5%

Only for top to show out of control CPU stats. Any advice appreciated at this stage. Thank you
Click to expand...
In /etc/systemd/system.control/user.slice.d/ you should have a number of files like

50-BlockIOAccounting.conf 50-CPUAccounting.conf 50-CPUShares.conf 50-MemoryAccounting.conf 50-TasksAccounting.conf

Create./change them using:

systemctl set-property user.slice BlockIOAccounting=yes

/*
BlockIOAccounting=yes
CPUAccounting=yes
CPUShares=1024
MemoryAccounting=yes
*/

Otherwise I would know what's different. It's been a while when I did it and it worked at once, so I didn't make any notes :)
 
I feel like I'm chiming in here out of place, having not really followed all of the recent discussion.

I do recall that when I tested this ... you have to actually log in as the under-privileged user. If you just su - that's not good enough, the slices still come from root. At least that's my recollection of this. Whether or not if this applies to the recent discussion, I'm not sure. Just thought I'd chime in and mention it.
 
sysdev said:
In /etc/systemd/system.control/user.slice.d/ you should have a number of files like

50-BlockIOAccounting.conf 50-CPUAccounting.conf 50-CPUShares.conf 50-MemoryAccounting.conf 50-TasksAccounting.conf

Create./change them using:

systemctl set-property user.slice BlockIOAccounting=yes

/*
BlockIOAccounting=yes
CPUAccounting=yes
CPUShares=1024
MemoryAccounting=yes
*/

Otherwise I would know what's different. It's been a while when I did it and it worked at once, so I didn't make any notes :)
Click to expand...

I will try this and report back.

sparek said:
I feel like I'm chiming in here out of place, having not really followed all of the recent discussion.

I do recall that when I tested this ... you have to actually log in as the under-privileged user. If you just su - that's not good enough, the slices still come from root. At least that's my recollection of this. Whether or not if this applies to the recent discussion, I'm not sure. Just thought I'd chime in and mention it.
Click to expand...

Oh, that has occurred. Thank you

The plan is to have all users under DirectAdmin managed via a plugin, individually. So they would have default values depending on package (assuming plugins can access package details, I'm not 100% sure) therefore everything that is outlined here is of great use and is pretty much a neat use case.

Thanks for stepping in, I'll post my findings asap.
 
Here is output on AlmaLinux with the above used. Unfortunately, still not working.

Code: 
[root@localhost ~]# ls /sys/fs/cgroup/
blkio    cpu,cpuacct  freezer  net_cls           perf_event  systemd
cpu      cpuset       hugetlb  net_cls,net_prio  pids
cpuacct  devices      memory   net_prio          rdma
craigfairhurst@Craigs-iMac-Pro-48 ~ % ssh -p 22 [email protected]
[email protected]'s password: 
Last failed login: Mon Apr 26 16:43:53 EDT 2021 from 192.168.1.50 on ssh:notty
There was 1 failed login attempt since the last successful login.
Last login: Mon Apr 26 16:43:04 2021
[root@localhost ~]# adduser test1
[root@localhost ~]# passwd test1
Changing password for user test1.
New password: 
BAD PASSWORD: The password is shorter than 8 characters
Retype new password: 
passwd: all authentication tokens updated successfully.
[root@localhost ~]# systemctl set-property user-1000.slice CPUQuota=5%
[root@localhost ~]# systemctl set-property user.slice BlockIOAccounting=yes
[root@localhost ~]# systemctl set-property user.slice CPUAccounting=yes
[root@localhost ~]# systemctl set-property user.slice CPUShares=1024
[root@localhost ~]# systemctl set-property user.slice MemoryAccounting=yes
[root@localhost ~]# systemctl set-property user-1000.slice CPUQuota=5%
[root@localhost ~]# id -u test1
1000
[root@localhost ~]# id -u rngd
994
[root@localhost ~]# systemctl set-property user-994.slice CPUQuota=5%
[root@localhost ~]# ls /sys/fs/cgroup/
cgroup.controllers      cgroup.subtree_control  init.scope
cgroup.max.depth        cgroup.threads          io.pressure
cgroup.max.descendants  cpu.pressure            memory.pressure
cgroup.procs            cpuset.cpus.effective   system.slice
cgroup.stat             cpuset.mems.effective   user.slice
[root@localhost ~]# ls /sys/fs/cgroup/user.slice
cgroup.controllers      cpu.weight       memory.oom.group
cgroup.events           cpu.weight.nice  memory.pressure
cgroup.freeze           io.bfq.weight    memory.stat
cgroup.max.depth        io.latency       memory.swap.current
cgroup.max.descendants  io.max           memory.swap.events
cgroup.procs            io.pressure      memory.swap.max
cgroup.stat             io.stat          pids.current
cgroup.subtree_control  memory.current   pids.events
cgroup.threads          memory.events    pids.max
cgroup.type             memory.high      user-0.slice
cpu.max                 memory.low       user-1000.slice
cpu.pressure            memory.max       user-994.slice
cpu.stat                memory.min
[root@localhost ~]# systemctl start user@1000
[root@localhost ~]# systemctl status user@1000
● [email protected] - User Manager for UID 1000
   Loaded: loaded (/usr/lib/systemd/system/[email protected]; static; vendor preset>
   Active: active (running) since Mon 2021-04-26 16:44:30 EDT; 4min 3s ago
 Main PID: 1975 (systemd)
   Status: "Startup finished in 71ms."
    Tasks: 2
   Memory: 528.0K
   CGroup: /user.slice/user-1000.slice/[email protected]
           └─init.scope
             ├─1975 /usr/lib/systemd/systemd --user
             └─1977 (sd-pam)
Apr 26 16:44:30 localhost.localdomain systemd[1975]: Starting D-Bus User Messag>
Apr 26 16:44:30 localhost.localdomain systemd[1975]: Listening on D-Bus User Me>
Apr 26 16:44:30 localhost.localdomain systemd[1975]: Reached target Sockets.
Apr 26 16:44:30 localhost.localdomain systemd[1975]: Reached target Basic Syste>
Apr 26 16:44:30 localhost.localdomain systemd[1975]: Reached target Default.
Apr 26 16:44:30 localhost.localdomain systemd[1975]: Startup finished in 71ms.
Apr 26 16:44:30 localhost.localdomain systemd[1]: Started User Manager for UID >
Apr 26 16:47:02 localhost.localdomain systemd[1975]: Starting Mark boot as succ>
Apr 26 16:47:03 localhost.localdomain systemd[1975]: grub-boot-success.service:>
Apr 26 16:47:03 localhost.localdomain systemd[1975]: Started Mark boot as succe>

[root@localhost ~]# ls /sys/fs/cgroup/user.slice/user-1000.slice/
cgroup.controllers      cgroup.type      memory.high          memory.swap.max
cgroup.events           cpu.max          memory.low           pids.current
cgroup.freeze           cpu.pressure     memory.max           pids.events
cgroup.max.depth        cpu.stat         memory.min           pids.max
cgroup.max.descendants  cpu.weight       memory.oom.group     session-4.scope
cgroup.procs            cpu.weight.nice  memory.pressure      [email protected]
cgroup.stat             io.pressure      memory.stat
cgroup.subtree_control  memory.current   memory.swap.current
cgroup.threads          memory.events    memory.swap.events
[root@localhost ~]# cat /sys/fs/cgroup/user.slice/user-1000.slice/cpu.max
5000 100000

Here you can see I have a change in my filesystem as described in the RHEL post here and my filesystem seems to be in order.. I must be missing a dependency, surely?

Kind regards
 
Do you have a configured slice for your user 1000 somewhat like below?

# pwd
/etc/systemd/system

# cat user-1000.slice
[Unit]
Description=1000 user.slice

[Slice]
CPUQuota=5%
CPUShares=1024
MemoryLimit=...
BlockIOReadBandwidth=/dev/mapper/cl-root ...
BlockIOWriteBandwidth=/dev/mapper/cl-root ...
TasksMax=1000

Edit:

After the systemctl set-property the values in:

/etc/systemd/system.control/user-1000.slice.d should change.
 
sysdev said:
Do you have a configured slice for your user 1000 somewhat like below?

# pwd
/etc/systemd/system

# cat user-1000.slice
[Unit]
Description=1000 user.slice

[Slice]
CPUQuota=5%
CPUShares=1024
MemoryLimit=...
BlockIOReadBandwidth=/dev/mapper/cl-root ...
BlockIOWriteBandwidth=/dev/mapper/cl-root ...
TasksMax=1000

Edit:

After the systemctl set-property the values in:

/etc/systemd/system.control/user-1000.slice.d should change.
Click to expand...


Here is my output:
Code: 
[root@localhost system]# ls /etc/systemd/system.control/user-1000.slice.d
50-CPUQuota.conf
[root@localhost system]# cat /etc/systemd/system.control/user-1000.slice.d/50-CPUQuota.conf 
# This is a drop-in unit file extension, created via "systemctl set-property"
# or an equivalent operation. Do not edit.
[Slice]
CPUQuota=5%
[root@localhost system]# systemctl set-property user-1000.slice CPUQuota=10%
[root@localhost system]# cat /etc/systemd/system.control/user-1000.slice.d/50-CPUQuota.conf 
# This is a drop-in unit file extension, created via "systemctl set-property"
# or an equivalent operation. Do not edit.
[Slice]
CPUQuota=10%

So yes it updates. I don't have anything under /etc/systemd/system relating to slices etc.
 
Just tried it on a clean CentOS Linux release 8.3.2011 DA machine:

1. add 'systemd.unified_cgroup_hierarchy=1' to GRUB_CMDLINE_LINUX in /etc/default/grub
2. run 'grub2-mkconfig -o /boot/grub2/grub.cfg'
2.a. option: check if /boot/grub2/grub.cfg has indeed the hierarchy option.
3. reboot
4. login as root in shell A
5. add a user 'test1' with 'adduser test1' and assign a password using 'passwd test1'. Remember the UID (e.g. 1010).
6. use a second shell (B) to login using the 'test1' user.
7. in shell B type 'dd if=/dev/zero of=/dev/null &' and press enter. Then start 'top' in the same shell (B). It should run at or near 100%.
8 switch to the root shell (A) and type 'systemctl set-property user-1010.slice CPUQuota=5%'. You should see the CPU% drop to 5% within seconds.

I did this exactly as written above on a clean DA setup and it worded on centos 8x. I really don't now why this wouldn't work on alma.
 
And magically, it's working. You have to set a password and login via command prompt.

This got me thinking, if you have to login via command prompt, will services under that account be able to max out as well?

Edit: I'll start on this in a couple weeks. Gotta get some other things done first.
 
youds said:
And magically, it's working. You have to set a password and login via command prompt.

This got me thinking, if you have to login via command prompt, will services under that account be able to max out as well?

Edit: I'll start on this in a couple weeks. Gotta get some other things done first.
Click to expand...
Logging in via the ssh starts the slice and any process started will have it's PID put in the cgroups pid list. These processes will be managed bij cgroup. Now... this does niet work for e.g. php-fpm as that is started as root, then spawns pools 'as the unpriviliged user', but these PIDS are not added in the correct cgroups PID list and will use all available power.

There are 2 main ways to handle this.

1. a 'wrong' way. This is not really wrong, but is inefficient. In this case you'll have to change a few config files and start a fpm process for each user. This is pretty easy as you can tapp into the scripts/custom/all.sh and handle switching php versions for a domain and use custom fpm@ templates. Then add the correct fpm line as a service and had that service take the PID of the started fpm-process and write that to the correct cgroup. I've don this as a test and with a few scripts it also creates nice mrtg graphs to be used in user/reseller plugins.
Even though you'll learn a lot from the setup, there's a more efficient way.

2. patch the php-fpm binary where you change the fpm_children.c file to also add the PID to the correct cgroups. I've mentioned that somewhere on the forum and this is also the path DA is going. Obviously this will only work for php related user processes.

So either way works, but the second is more efficient. At least for php-fpm. In the end you'll have shell logins and php managed and this will count for 90% of your load... most of the time...
You still have mysql/mariabd... you can limit user queries etc per user, but you don't want to do that as queries are executed in mysql's scope and that pretty high priority. So you would need to monitor and manage database threads for each user (kinda like the mysql governor of cloudlinux). I'm not sure if you'd want to do this.
 
youds said:
And magically, it's working. You have to set a password and login via command prompt.

This got me thinking, if you have to login via command prompt, will services under that account be able to max out as well?

Edit: I'll start on this in a couple weeks. Gotta get some other things done first.
Click to expand...
Yea, this is what I discovered when I first worked with this - many moons ago.

It turned out to not be very viable, because the individual user slices weren't being used in php-fpm processes.

But I do recall someone (probably @sysdev) mentioning a patch for PHP to account for this. And some where I thought I saw where DirectAdmin was incorporating this patch - at least in some capacity (might've been in a beta channel or something). So that may involve reinvestigating this procedure.

The difficult part for me is that I'm not using DirectAdmin's Custombuild for PHP. Instead I'm using the Remi repository for PHP because I don't like recompiling PHP every month when a new release comes out. The hope would be that whatever patch is applied to PHP for fpm processes would be applied up source by PHP.
 
You must log in or register to reply here.
Back
Top