Troubleshooting an issue in your software made impossible

[CodeAssistant]

We're experiencing an issue on one of our servers, where going to MySQL Management doesn't work:

The error matches an error shown when trying to make a backup:

2025/03/25 11:20:46 error getting databases list    error=Error 1577 (HY000): Cannot proceed, because event scheduler is disabled req-uri=/CMD_DB?domain={redacted} req-user=admin|{redacted}

Database: listing user databases: Error 1577 (HY000): Cannot proceed, because event scheduler is disabled

Event-scheduler is indeed disabled, but it's disabled on all of our (hundreds of) other servers as well, so this cannot be the actual cause of the problems.
We're trying to figure out what the actual root-cause is by debugging DirectAdmin, but even in the debug log it shows nothing but that same error.
For our debugging process, we've had to restart DirectAdmin several times, and are now faced with this error:

Needless to say, I'm starting to get pretty livid, because not only is your software failing to function properly, it's also failing to allow me a normal/smooth debugging process.
It's absolutely unacceptable that our customers are not only faced with the initial issue, but downtime of their administrator panel due to you blocking the entire licensing after a few repeat restarts of DA which are necessary due to your software's incomplete error logging/debug output.

At this point, please just tell us what causes this issue, and don't lie to us and say it's caused by the event-scheduler being disabled.
Also, don't tell us to restart MySQL, because I know this would (temporarily) resolve the issue, it doesn't tell us anything about the cause and restarting MySQL on an actively used server isn't an acceptable workaround.

 

[CodeAssistant]

We've now discovered how to resolve this issue without a reboot.
We figured out the issue is a result of the following underlying issue:

MariaDB [(none)]>  SET GLOBAL event_scheduler = ON;
ERROR 1408 (HY000): Event Scheduler: An error occurred when initializing system tables. Disabling the Event Scheduler.
MariaDB [(none)]>

We then updated the mysql.event table with the correct parameters which should've already been done by mysql_upgrade in the past.
This resulted in a new issue:

2025/03/25 13:14:58 error getting databases list    error=Error 1728 (HY000): Cannot load from mysql.proc. The table is probably corrupted req-uri=/CMD_DB?domain={redacted} req-user=admin|{redacted}

Which was resolved after re-running mysql_upgrade.

However, this doesn't answer two questions:
- How come this issue randomly appeared, months after MySQL was upgraded?
- How come this issue can also be resolved by simply restarting MySQL?