MySQL does not start

Abilnet

Verified User
Joined
Apr 21, 2010
Messages
15
Location
Sunny Spain
My server has been working without any problems for years, until now. What happened:

* after updating some modules to my Drupal CMS, I'd run "update.php" (Drupal) to update database tables (this is a very routine & standard procedure in Drupal)
* ended up to error -messages from CMS
* realized, that the database of the site in question was somehow corrupted
* backups did not work, errors continued
* at this point, other sites on the server were still working ok (obviously MySQL was up & running)
* restarted MySQL (trying to solve the problematic database issue)
* MySQL did not start anymore (and is still dead)

Done since:

* read a lot of forums & Google
* verified hard disks; a lot of space available
* even rebooted the entire box (sorry about that)
* trying to restart MySQL will output:

Code:
root@server1:~# /etc/init.d/mysqld restart
Stopping mysqld:		[ FAILED ]
Starting mysqld:		[ OK ]
root@server1:~#

* from my CMS I'm getting errors:

Code:
PDOException: SQLSTATE[HY000] [2002] Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2) in lock_may_be_available() (line 167 of /home/xxxxxx/domains/xxxxx.com/public_html/includes/lock.inc).

AND

ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)

* error from phpMyAdmin:

Code:
#2002 - The server is not responding (or the local MySQL server's socket is not correctly configured)

* my /etc/mysql/my.cnf:

Code:
[client]
port            = 3306
# socket        = /var/run/mysqld/mysqld.sock
socket          = /tmp/mysql.sock

[mysqld_safe]
# socket        = /var/run/mysqld/mysqld.sock
socket          = /tmp/mysql.sock
nice            = 0

[mysqld]
user            = mysql
# socket        = /var/run/mysqld/mysqld.sock
socket          = /tmp/mysql.sock
port            = 3306
basedir         = /usr
# datadir       = /var/lib/mysql
datadir         = /home/mysql
tmpdir          = /tmp
skip-external-locking

bind-address            = 127.0.0.1

key_buffer              = 16M
max_allowed_packet      = 16M
thread_stack            = 192K
thread_cache_size       = 8
myisam-recover         = BACKUP
query_cache_limit       = 1M
query_cache_size        = 16M

log_error                = /var/log/mysql/error.log

expire_logs_days        = 10
max_binlog_size         = 100M
skip-innodb

[mysqldump]
quick
quote-names
max_allowed_packet      = 16M

[mysql]
no-auto-rehash  # faster start of mysql but no tab completition

[isamchk]
key_buffer              = 16M

!includedir /etc/mysql/conf.d/

* /home/mysql/s1.server.com.err:

Code:
101204 06:37:36 mysqld_safe Starting mysqld daemon with databases from /usr/local/mysql/data
InnoDB: The first specified data file ./ibdata1 did not exist:
InnoDB: a new database to be created!
101204  6:37:36  InnoDB: Setting file ./ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
101204  6:37:37  InnoDB: Log file ./ib_logfile0 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile0 size to 5 MB
InnoDB: Database physically writes the file full: wait...
101204  6:37:37  InnoDB: Log file ./ib_logfile1 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile1 size to 5 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
101204  6:37:37  InnoDB: Started; log sequence number 0 0
101204  6:37:37 [Note] Event Scheduler: Loaded 0 events
101204  6:37:37 [Note] /usr/local/mysql/bin/mysqld: ready for connections.
Version: '5.1.34'  socket: '/tmp/mysql.sock'  port: 3306  MySQL Community Server (GPL)
101206 13:57:54 [Note] /usr/local/mysql/bin/mysqld: Normal shutdown

101206 13:57:54 [Note] Event Scheduler: Purging the queue. 0 events
101206 13:57:54  InnoDB: Starting shutdown...
101206 13:57:56  InnoDB: Shutdown completed; log sequence number 0 84114
101206 13:57:56 [Warning] Forcing shutdown of 1 plugins
101206 13:57:56 [Note] /usr/local/mysql/bin/mysqld: Shutdown complete

101206 13:57:56 mysqld_safe mysqld from pid file /usr/local/mysql/data/s1.pid ended
101206 13:58:52 mysqld_safe Starting mysqld daemon with databases from /usr/local/mysql/data
101206 13:58:53  InnoDB: Started; log sequence number 0 84114
101206 13:58:53 [Note] Event Scheduler: Loaded 0 events
101206 13:58:53 [Note] /usr/local/mysql/bin/mysqld: ready for connections.
Version: '5.1.34'  socket: '/tmp/mysql.sock'  port: 3306  MySQL Community Server (GPL)
101208 13:24:30 [Note] /usr/local/mysql/bin/mysqld: Normal shutdown

101208 13:24:30 [Note] Event Scheduler: Purging the queue. 0 events
101208 13:24:30  InnoDB: Starting shutdown...
101208 13:24:31  InnoDB: Shutdown completed; log sequence number 0 84114
101208 13:24:31 [Warning] Forcing shutdown of 1 plugins
101208 13:24:31 [Note] /usr/local/mysql/bin/mysqld: Shutdown complete
101210 10:21:52 mysqld_safe mysqld from pid file /usr/local/mysql/data/s1.pid ended

* datadir = /var/lib/mysql:
---> directory does not exist (data seems (?) to be in: /home/mysql)

* socket = /var/run/mysqld/mysqld.sock:
---> does not exist, not even the directory "mysql"

* MySQL does not start also from Direct Admin, only an error message: "An error has occurred" (with no additional details)

* my server setup:

Code:
Ubuntu 10.04 LTS 
DirectAdmin 1.40.1
MySQL 5.1.34
Php 5.2.14
Apache 2.2.17

I've tried everything possible I can think of, your ideas & help Very Much appreciated. Thank you.
 
Hello,

Try to remove /tmp/mysql.sock and restart MySQL
if it does not help, comment all lines in /etc/my.cnf containing /tmp/mysql.sock and restart MySQL.
 
Thank you Alex for your answer, appreciated.

However, unfortunately the tips did not make any difference. Actually there was not even a trace to "mysql.sock" in the "/tmp" -directory. When starting or restarting mysql (after commenting out socket path "socket = /tmp/mysql.sock" in "/etc/mysql/my.cnf") the behavior is the same than earlier (mysql seems to be starting, but actually does not do it)

Code:
root@server1:/etc/mysql# /etc/init.d/mysqld start
Starting mysqld:		[ OK ]
root@server1:/etc/mysql# /etc/init.d/mysqld restart
Stopping mysqld:		[ FAILED ]
Starting mysqld:		[ OK ]
root@server1:/etc/mysql#

Also Direct Admin is reporting the same than earlier.

Any additional ideas highly appreciated. Thank you.
 
I did not see any critical error in your error log...

Code:
ps aux | grep -v grep | grep mysql

Does it show anything?
 
* ps aux | grep -v grep | grep mysql:

Code:
root@server1:/etc/mysql# ps aux | grep -v grep | grep mysql
root      2841  0.0  0.0   2756  1264 ?        S    10:26   0:00 /bin/sh /usr/local/mysql/bin/mysqld_safe --user=mysql --datadir=/usr/local/mysql/data --pid-file=/usr/local/mysql/data/server1.pid
mysql     2960  0.0  0.3 110800 15688 ?        Sl   10:26   0:00 /usr/local/mysql/bin/mysqld --basedir=/usr --datadir=/usr/local/mysql/data --user=mysql --log-error=/usr/local/mysql/data/server1.xxxxxxx.com.err --pid-file=/usr/local/mysql/data/server1.pid --port=3306
root@server1:/etc/mysql#

I did not see any critical error in your error log...

Some additional logs:

* Direct Admin Task Queue Log /var/log/directadmin/errortaskq.log

Code:
2012:01:29-10:08:01: service mysqld wasn't running, starting it
2012:01:29-10:08:11: service mysqld didn't start, re-starting it
2012:01:29-10:09:01: service mysqld wasn't running, starting it
2012:01:29-10:09:11: service mysqld didn't start, re-starting it
2012:01:29-10:10:01: service mysqld wasn't running, starting it
2012:01:29-10:10:11: service mysqld didn't start, re-starting it
2012:01:29-10:11:01: service mysqld wasn't running, starting it
2012:01:29-10:11:11: service mysqld didn't start, re-starting it
2012:01:29-10:12:01: service mysqld wasn't running, starting it
2012:01:29-10:12:11: service mysqld didn't start, re-starting it
2012:01:29-10:13:01: service mysqld wasn't running, starting it
2012:01:29-10:13:11: service mysqld didn't start, re-starting it
2012:01:29-10:14:01: service mysqld wasn't running, starting it
2012:01:29-10:14:11: service mysqld didn't start, re-starting it
2012:01:29-10:15:01: service mysqld wasn't running, starting it
2012:01:29-10:15:11: service mysqld didn't start, re-starting it
2012:01:29-10:16:01: service mysqld wasn't running, starting it
2012:01:29-10:16:11: service mysqld didn't start, re-starting it
2012:01:29-10:17:01: service mysqld wasn't running, starting it
2012:01:29-10:17:11: service mysqld didn't start, re-starting it
2012:01:29-10:18:01: service mysqld wasn't running, starting it
2012:01:29-10:18:11: service mysqld didn't start, re-starting it
2012:01:29-10:19:01: service mysqld wasn't running, starting it
2012:01:29-10:19:11: service mysqld didn't start, re-starting it
2012:01:29-10:20:01: service mysqld wasn't running, starting it
2012:01:29-10:20:11: service mysqld didn't start, re-starting it
2012:01:29-10:21:02: service mysqld wasn't running, starting it
2012:01:29-10:21:12: service mysqld didn't start, re-starting it
2012:01:29-10:22:01: service mysqld wasn't running, starting it
2012:01:29-10:22:11: service mysqld didn't start, re-starting it
2012:01:29-10:23:01: service mysqld wasn't running, starting it
2012:01:29-10:23:11: service mysqld didn't start, re-starting it
2012:01:29-10:24:01: service mysqld wasn't running, starting it
2012:01:29-10:24:11: service mysqld didn't start, re-starting it
2012:01:29-10:25:01: service mysqld wasn't running, starting it
2012:01:29-10:25:11: service mysqld didn't start, re-starting it
2012:01:29-10:26:01: service mysqld wasn't running, starting it
2012:01:29-10:26:11: service mysqld didn't start, re-starting it
2012:01:29-10:27:01: service mysqld wasn't running, starting it
2012:01:29-10:27:11: service mysqld didn't start, re-starting it
2012:01:29-10:28:01: service mysqld wasn't running, starting it
2012:01:29-10:28:11: service mysqld didn't start, re-starting it
2012:01:29-10:29:01: service mysqld wasn't running, starting it
2012:01:29-10:29:11: service mysqld didn't start, re-starting it
2012:01:29-10:30:01: service mysqld wasn't running, starting it
2012:01:29-10:30:11: service mysqld didn't start, re-starting it
2012:01:29-10:31:01: service mysqld wasn't running, starting it
2012:01:29-10:31:11: service mysqld didn't start, re-starting it
2012:01:29-10:32:01: service mysqld wasn't running, starting it
2012:01:29-10:32:11: service mysqld didn't start, re-starting it

* Direct Admin System Log /var/log/directadmin/system.log

Code:
2012:01:29-10:13:01: mysqld started
2012:01:29-10:13:11: mysqld restarted
2012:01:29-10:14:01: mysqld started
2012:01:29-10:14:11: mysqld restarted
2012:01:29-10:15:01: mysqld started
2012:01:29-10:15:11: mysqld restarted
2012:01:29-10:16:01: mysqld started
2012:01:29-10:16:11: mysqld restarted
2012:01:29-10:17:01: mysqld started
2012:01:29-10:17:11: mysqld restarted
2012:01:29-10:18:01: mysqld started
2012:01:29-10:18:11: mysqld restarted
2012:01:29-10:19:01: mysqld started
2012:01:29-10:19:11: mysqld restarted
2012:01:29-10:20:01: mysqld started
2012:01:29-10:20:11: mysqld restarted
2012:01:29-10:21:02: mysqld started
2012:01:29-10:21:12: mysqld restarted
2012:01:29-10:22:01: mysqld started
2012:01:29-10:22:11: mysqld restarted
2012:01:29-10:23:01: mysqld started
2012:01:29-10:23:11: mysqld restarted
2012:01:29-10:24:01: mysqld started
2012:01:29-10:24:11: mysqld restarted
2012:01:29-10:25:01: mysqld started
2012:01:29-10:25:11: mysqld restarted
2012:01:29-10:26:01: mysqld started
2012:01:29-10:26:11: mysqld restarted
2012:01:29-10:27:01: mysqld started
2012:01:29-10:27:11: mysqld restarted
2012:01:29-10:28:01: mysqld started
2012:01:29-10:28:11: mysqld restarted
2012:01:29-10:29:01: mysqld started
2012:01:29-10:29:11: mysqld restarted
2012:01:29-10:30:01: mysqld started
2012:01:29-10:30:11: mysqld restarted
2012:01:29-10:31:01: mysqld started
2012:01:29-10:31:11: mysqld restarted
2012:01:29-10:32:01: mysqld started
2012:01:29-10:32:11: mysqld restarted
2012:01:29-10:33:01: mysqld started
2012:01:29-10:33:11: mysqld restarted
2012:01:29-10:34:01: mysqld started
2012:01:29-10:34:11: mysqld restarted
2012:01:29-10:35:01: mysqld started
2012:01:29-10:35:11: mysqld restarted
2012:01:29-10:36:01: mysqld started
2012:01:29-10:36:11: mysqld restarted
2012:01:29-10:37:01: mysqld started
2012:01:29-10:37:11: mysqld restarted

Sorry being a newbie, but is there some other logs I should take a look?

Thanks again Alex for your help.
 
Last edited:
You've got a running MySQL server... so you should kill it and start again.

Thanks Alex again.

Tried to find the proper command to kill mysqld, but was not sure... I decided to test the advice from Alex by doing a dirty trick by entirely hiding "my.cnf" from "/etc/mysql/"

This seems to partially work, and I now have mysqld up and running in "safe mode"

Code:
root@server1:/etc/mysql# ls -la
total 20
drwxr-xr-x  3 root root 4096 2012-01-29 11:56 .
drwxr-xr-x 95 root root 4096 2012-01-29 00:10 ..
drwxr-xr-x  2 root root 4096 2011-02-10 08:53 conf.d
-rw-r--r--  1 root root 3686 2012-01-29 09:12 my.cnf__MOD
-rw-r--r--  1 root root 3564 2012-01-28 12:34 my.cnf__ORIG
root@server1:/etc/mysql# /etc/init.d/mysqld restart
Stopping mysqld:		[ OK ]
Starting mysqld:		[ OK ]
root@server1:/etc/mysql# 120129 12:00:32 mysqld_safe A mysqld process already exists
^C
root@server1:/etc/mysql#

...though I still have no clear idea, how to reach the normally working situation. Help still Highly appreciated. Thank you.
 
Last edited:
Code:
man kill
Code:
mysqladmin --help

I decided to test the advice from Alex by doing a dirty trick by entirely hiding "my.cnf" from "/etc/mysql/"

I did not suggest anything of this kind.
 
From your code post:
Code:
root@server1:/etc/mysql# /etc/init.d/mysqld restart
This should be sufficient to restart MySQLd. If you're still having problems with my.cnf in place, and running without my.cnf in place, then the problem resides somewhere in your my.cnf file.

Jeff
 
This should be sufficient to restart MySQLd.

Not always. I've faced an issue several times when MySQL can not be stopped this usual way. For example if you've got some "zombie connections" or "locked processes"... or socket error with a running copy of mysqld. In this case mysqladmin helps to stop all connections and to completely stop mysql.
 
Hi,

I wiill use this thread for my question. Im installing clean DA and now with custombuild I wan to install mysql 5.5 and I get this error:

Linking CXX shared module ha_federated.so
[100%] Built target federated
Make complete
Installing mysql-5.5.27...
CPack: Create package using TGZ
CPack: Install projects
CPack: - Run preinstall target for: MySQL
CPack: - Install project: MySQL
CPack: Create package
CPack: - package: /usr/local/directadmin/custombuild/mysql-5.5.27/mysql-5.5.27-freebsd8.3-x86_64.tar.gz generated.
Moving mysql-5.5.27-*.tar.gz to main direcotry...
Done.
Stopping mysqld ...
Stopping mysqld: [ FAILED ]
Installing mysql-5.5.27-freebsd8.3-x86_64.tar.gz...
mysql-5.5.27-freebsd8.3-x86_64/data/test/dummy.bak: Cannot extract through symlink mysql-5.5.27-freebsd8.3-x86_64/data
mysql-5.5.27-freebsd8.3-x86_64/data/mysql/dummy.bak: Cannot extract through symlink mysql-5.5.27-freebsd8.3-x86_64/data
tar: Error exit delayed from previous errors.
Starting mysqld: [ OK ]
Giving mysqld a few seconds to start up...
Looking for 'mysql' as: /usr/local/mysql/bin/mysql
Looking for 'mysqlcheck' as: /usr/local/mysql/bin/mysqlcheck
Running 'mysqlcheck with default connection arguments
/usr/local/mysql/bin/mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2) when trying to connect
FATAL ERROR: Upgrade failed
Done mysql-5.5.27-freebsd8.3-x86_64.
Restarting MySQL.
Stopping mysqld: [ FAILED ]
Starting mysqld: [ OK ]
It is recommended to do "./build php n" after MySQL update.

Any tip?


Thanks
Regards
 
mysql-5.5.27-freebsd8.3-x86_64/data/test/dummy.bak: Cannot extract through symlink mysql-5.5.27-freebsd8.3-x86_64/data
mysql-5.5.27-freebsd8.3-x86_64/data/mysql/dummy.bak: Cannot extract through symlink mysql-5.5.27-freebsd8.3-x86_64/data

mysql does not start....
 
If you look more close you will see this

Stopping mysqld ...
Stopping mysqld: [ FAILED ]
Installing mysql-5.5.27-freebsd8.3-x86_64.tar.gz...
mysql-5.5.27-freebsd8.3-x86_64/data/test/dummy.bak: Cannot extract through symlink mysql-5.5.27-freebsd8.3-x86_64/data
mysql-5.5.27-freebsd8.3-x86_64/data/mysql/dummy.bak: Cannot extract through symlink mysql-5.5.27-freebsd8.3-x86_64/data
tar: Error exit delayed from previous errors.
Starting mysqld: [ OK ]
Giving mysqld a few seconds to start up...
Looking for 'mysql' as: /usr/local/mysql/bin/mysql
Looking for 'mysqlcheck' as: /usr/local/mysql/bin/mysqlcheck
Running 'mysqlcheck with default connection arguments
/usr/local/mysql/bin/mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2) when trying to connect
FATAL ERROR: Upgrade failed
Done mysql-5.5.27-freebsd8.3-x86_64.
Restarting MySQL.
Stopping mysqld: [ FAILED ]

MySQL is not running....
 
Search the forums then to learn how to deal with MySQL error log in order to find reasons why MySQL fails to start.
 
Back
Top