$#!tstemd cgroups CPU, RAM, IO, (TASKs?) made easy

[sysdev]

For an example of some cg tools: https://www.server-world.info/en/note?os=CentOS_7&p=cgroups&f=1

 

[youds]

Yeah, my DA installation has been borked all week. Support are looking at it, I paid for LAN support and troubleshooting so it should get somewhere. Just means the backup plug-in is on for next week, which is when I planned to do this. No bother just might be a bit delayed.

Keep the useful info coming

 

[sysdev]

I use a very simple version where I user custom package options to set cpu, mem, tasks and bandwidth limits.

In user_create_post.sh I write the slice info to /etc/systemd/system/user-NNNN.slice.
In user_destroy_pre.sh I obviously remove the file again.
In user_modify_post.sh I update the user-NNNN.slice file and run a a few "systemctl set-property user-${userID}.slice CPUQuota=${resCPU}%" commands to activate the changes directly.
After each script I check if the daemon needs to be reloaded and user.slice needs to restart.

Pretty straightforward stuff.

What would be nice is some way to present usage graphs so it's easier to see if a user needs more resources.

 

[sysdev]

Actually, I use MRTG now to show the users load against his cpu limit.

One thing that kinda bothers me is the reseller. If e.g. a reseller has limits like 100% cpu ( = one core ) and 1G ram, then it would be silly if he could assign 2G ram to one of his users. The same goes for assigning 512M to maybe ten of his users totalling 5G.
Rescheduling each user to a max of 1G / # users would give each user 100M... not good resources management either.

Any ideas on how to handle this would be welcome

 

[k1l0b1t]

I use a very simple version where I user custom package options to set cpu, mem, tasks and bandwidth limits.

In user_create_post.sh I write the slice info to /etc/systemd/system/user-NNNN.slice.
In user_destroy_pre.sh I obviously remove the file again.
In user_modify_post.sh I update the user-NNNN.slice file and run a a few "systemctl set-property user-${userID}.slice CPUQuota=${resCPU}%" commands to activate the changes directly.
After each script I check if the daemon needs to be reloaded and user.slice needs to restart.

Pretty straightforward stuff.

What would be nice is some way to present usage graphs so it's easier to see if a user needs more resources.

Would you like to share how you did this?

 

[sparek]

I got a chance to look into this a little bit more.

Unfortunately, this doesn't work with php-fpm. The user-slice information isn't read into the php-fpm pool fork per user. The user-slick information is only read from processes that originally start out from that user. Since the majority of process usage in the webhosting arena is via web and via PHP - this would appear to be a non-starter for me.

Unless I'm missing something from the systemd aspect that will allow the user-slice information to be read when the php-fpm process forks out to individual user pools.

Ideally the user-slices would be fully templateable. That's what cgroup templates used to do. Any process run by a user fell under the jurisdiction of that user's cgroup template. This does not appear to be the case with user-slices.

 

[sysdev]

I got a chance to look into this a little bit more.

Unfortunately, this doesn't work with php-fpm. The user-slice information isn't read into the php-fpm pool fork per user. The user-slick information is only read from processes that originally start out from that user. Since the majority of process usage in the webhosting arena is via web and via PHP - this would appear to be a non-starter for me.

Unless I'm missing something from the systemd aspect that will allow the user-slice information to be read when the php-fpm process forks out to individual user pools.

Ideally the user-slices would be fully templateable. That's what cgroup templates used to do. Any process run by a user fell under the jurisdiction of that user's cgroup template. This does not appear to be the case with user-slices.

Well, not with the default user slice but that's only for users logging in using ssh e.g.
For php-fpm you'll need to do something else, but that's not that hard either. Systemd uses templates like 'php-fpm73@.service'. You'll need toch use this to start the 'root' fpm-php process for each user. This service can run a post script like:

[Service]
ExecStartPost=/blabla/cg_user.sh %i 73

This cg_user.sh script gets the username and php version as a parameter. You can use that to get the cgroup settings from his package and set them in /sys/fs/cgroup/user.slice/<userid>/cpu.max etc.

 

[smtalk]

Well, not with the default user slice but that's only for users logging in using ssh e.g.
For php-fpm you'll need to do something else, but that's not that hard either. Systemd uses templates like 'php-fpm73@.service'. You'll need toch use this to start the 'root' fpm-php process for each user. This service can run a post script like:

[Service]
ExecStartPost=/blabla/cg_user.sh %i 73

This cg_user.sh script gets the username and php version as a parameter. You can use that to get the cgroup settings from his package and set them in /sys/fs/cgroup/user.slice/<userid>/cpu.max etc.

It's been discussed in https://github.com/php/php-src/pull/2440 and https://bugs.php.net/bug.php?id=70605. There is little activity there, so.. maybe these need to be 'revived' ?

 

[sysdev]

It's been discussed in https://github.com/php/php-src/pull/2440 and https://bugs.php.net/bug.php?id=70605. There is little activity there, so.. maybe these need to be 'revived' ?

Well, how I've done it, works good enough for me because I don't have to change the php code. (I still once in a while have nightmares about the apache itk mod). By moving the root php-fpm process (one pid only used when the fpm service is started) will have all subsequent user fpm processes also running in this cgroup automatically. The only 'issue' is that I have a php-fpm root process per user.
But if cgroup support for pools would become mainstream i'd adopt it asap. But for now I'm happy with having it being part of an DA package like:

 

[sysdev]

Just took a quick look at the php source code and I'm wondering if a simple solution would work to change the fpm_children.c function fpm_children_make() where a child is forked and after a successful fork simply write the child pid to the required cgroup.procs. That would prevent the need to have a master process per user. Something like:

child->pid = pid;
/* Added: Move pid to cgroup */
sprintf(str, "/sys/fs/cgroup/user.slice/%s/cgroup.procs", wp->config->name );
destFile = fopen(str, "w+");
fwrite(destFile, pid);
fclose(destFile);
/* EOA: Move pid to cgroup */
fpm_clock_get(&child->started);
fpm_parent_resources_use(child);

zlog(is_debug ? ZLOG_DEBUG : ZLOG_NOTICE, "[pool %s] child %d started", wp->config->name, (int) pid);