Intermittent error: The service 'lfd' on server myserver.domain.com is currently down

Roberto

Roberto

Verified User
Joined
Apr 6, 2013
Messages
127
Location
London
Hi

Yesterday afternoon, I used CustomBuild 2.0 to install the new updates. I think one of them was PHP related. I also ran yum update from the prompt. Yesterday evening and today I get intermittent errors concerning lfd:-

Code: 
“The service 'lfd' on server myserver.domain.com is currently down”

From within CSF, Show lfd status...

Code: 
● lfd.service - ConfigServer Firewall & Security - lfd
   Loaded: loaded (/usr/lib/systemd/system/lfd.service; enabled; vendor preset: disabled)
   Active: activating (start) since Wed 2019-09-25 14:21:36 BST; 53s ago
  Process: 27709 ExecStart=/usr/sbin/lfd (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/lfd.service
           └─27755 lfd - sleepin

Sep 25 14:21:36 server systemd[1]: Starting ConfigServer Firewall & Security - lfd...
Sep 25 14:21:37 server systemd[1]: Can't open PID file /var/run/lfd.pid (yet?) after start: Too many levels of symbolic links


Does anyone have any idea why I’m getting this all of a sudden? The VPS server has been running for years without this particular error, as far as I can recall. Is it connected to the recent updates? Is anyone else experiencing this yesterday and today after updating?

Any advice most welcome?
 
This one after kernel update?

error "Too many levels of symbolic links". This is an upstream issue involving the latest update to systemd included in CentOS 7.7.

Technical Summary
The recent CentOS 7.7 update includes an update to systemd-219-67, which changed how systemd opens symbolic links. If your kernel doesn’t support the new method which systemd uses to open symbolic links, then you may see “Too many levels of symbolic links” errors as seen in the example below:
Click to expand...

https://forum.configserver.com/viewtopic.php?t=11485&p=30845
 
Last edited:
Hi ikkeben

yes, now that you mention it when I ran the command on CentOS "yum update -y", one of the updates was a kernel update.

I am still as yet unsure how to solve this. Should I simply roll back to a previous kernel version? Any ideas?

Regards
 
Update...

I've requested my VPS providers carry out workaround 1 from the website I mentioned in my previous post.

Workaround 1 (requires access to the VPS hardware node)
1) Update the Virtuozzo kernel on the server.
2) Reboot the server.

I'll post back when I learn more.
 
This is the response I got from the VPS provider. (redacted content which relates to names)

Hello {my name},

Thank you for reaching out to {company name}.
Unfortunately, we can not update kernel as there are other VPSs on the node
which would interfere with everyone. Please create the backup for your data and
try reverting the changes you have made.
Thank you for your cooperation.

Not happy with that response. I'm fairly certain that others VPS users would want to be running the latest
version of Virtuozzo kernel on the server if that meant no problems.
 
Dit you saw this workaround ?:

Workaround 2
Create an addon service file for each service that is failing in order to change the location of the PID file. Using "spamd" as an example:

1) Execute the following command via SSH as a user with root access:

systemctl edit spamd.service

2) The command will bring up a text editor, at which point you can add entries like the ones below to modify where the PID file is located:

[Service]

PIDFile=/run/spamd.pid

Exit the text editor after adding the entries and save the changes you made. Make sure you change this content to match different services as needed.

3) Confirm the custom override file for systemd was automatically populated:


# cat /etc/systemd/system/spamd.service.d/override.conf

[Service]

PIDFile=/run/spamd.pid

4) After making any changes, reload systemd and the failed service. EX:

systemctl daemon-reload
/scripts/restartsrv_spamd

more here:
https://forums.cpanel.net/threads/ups-177-service-failures-after-centos-7-7-update.659933/
 
Hi Active8

Thanks for your response. Yes, I noticed workaround #2. I planned on putting this aside until tomorrow. I want to understand what is being achieved before i make any changes, so needed a little more time to research it.

I'll post back once I look into it further.

Regards
 
Just out of interest, are you only using specific VPS systems?

We're running a couple of dedicated servers with Centos 7 and also did the Centos upgrade to 7.7 but did not encounter any of these spamd or csf issues yet.

We also have a VPS running which is updated (don't know which vps software) to Centos 7.7 also without any issues.
 
Hi Richard G
The node this particular VPS instance is on is running OpenVZ. I issued the yum update command which updated Centos as mentioned, which is not compatible with the slightly outdated kernel running on that box. After many messages back and forth the providers still refuse to update the kernel on their end. They keep repeating their mantra that I should upgrade to their hybrid servers and pay much much more. I'd rather move than pay for something I do not need.
If you're on a dedicated server then presumably this would not affect you, because you have the control to update the kernel yourself. As for the VPS server you're running, I guess that those too are up-to-date on the back end, hence you have not encoutered this. On the link that i previous posted...

https://forums.cpanel.net/threads/ups-177-service-failures-after-centos-7-7-update.659933/

... it states that affected systems include

• CentOS 7.7 and OpenVZ/Virtuozzo with kernels older than 2.6.32-042stab134.7
• CentOS 7.7 with kernel older than 2.6.39

Thanks
 
Update

By the way, I did Workaround #2 as described in the link and changed the example from

systemctl edit spamd.service

to

systemctl edit lfd.service

... replacing spamd with lfd each step of the workaround instructions.
 
Hello themadguru
I'd rather move than pay for something I do not need.
Click to expand...
I quite agree with you.
Next to our dedicated servers, we do also have 2 VPS systems running. But I don't know how to detect which VPS system is running on them. They both have done kernel updates however, and I don't have this issue on either VPS.

So that's why I'm wondering if this issue might be limited to a certain VPS system like OpenVZ/Virtuozzo. I did read the link you posted. But I've seen the kernel being updated on both VPS systems when I did the yum update, so I presume the vps host did not have to update it?

On my VPS it says:
Code: 
[root@server: ~]# lscpu|grep Hypervisor
Hypervisor vendor:     KVM
No idea on what's running on there but it's fully configurable according to the host.
The other VPS only gives an error on that command.
lscpu: failed to determine number of CPUs: /sys/devices/system/cpu/possible: No such file or directory
Click to expand...
 
Hi Richard

Richard G said:
Hello themadguru
No idea on what's running on there but it's fully configurable according to the host.
The other VPS only gives an error on that command.
Click to expand...

I came across this page:

https://www.cyberciti.biz/faq/linux-determine-virtualization-technology-command/

Which suggests you can install a package called virt-what to determine what virtualization technology the node your VPS is running on.

$ sudo yum install virt-what
Click to expand...

I hope this helps.

Regards
 
Hello themadguru.

Oh the other VPS that package was already installed, I didn't even know it, but it just said KVM, so lscpu gave mor info since that also showed Hypervisor.

The unknown one is now known due to your tip. It's OpenVZ. Kernel 3.10.0-042stab140.1.
 
You must log in or register to reply here.
Back
Top