SQLSTATE[HY000] [2002] Connection refused #3406

roswitina · 2020-03-11T12:47:20Z

Description of the bug:
The system no longer starts after an update.

System docker-compose Attaching mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 |
mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 mphp-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 |
php-fpm-mailcow_1 php-fpm-mailcow_1 | Stack trace:
php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 php-fpm-mailcow_1 information:
logs --tail=200 php-fpm-mailcow
to mailcowdockerized_php-fpm-mailcow_1
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Uptime: 1 Threads: 8 Questions: 1 Slow queries: 0 Opens: 17 Flush tables: 1 Open tables: 11 Queries per second avg: 1.000
| We are master, preparing...
| MySQL @ 8e1cb5c0ec328fe922c23b489e7eb4a6183587385852f3f402a7103979b5778a
| SQL upgrade iteration #1
| No valid reponse for mysql_upgrade was received, debug output:
| {"msg":"409 Client Error: Conflict ("Container 8e1cb5c0ec328fe922c23b489e7eb4a6183587385852f3f402a7103979b5778a is restarting, wait until the container is running")","type":"danger"}
| SQL upgrade iteration #2
| No valid reponse for mysql_upgrade was received, debug output:
| {"msg":"409 Client Error: Conflict ("Container 8e1cb5c0ec328fe922c23b489e7eb4a6183587385852f3f402a7103979b5778a is restarting, wait until the container is running")","type":"danger"}
| SQL upgrade iteration #3
| No valid reponse for mysql_upgrade was received, debug output:
| {"msg":"409 Client Error: Conflict ("Container 8e1cb5c0ec328fe922c23b489e7eb4a6183587385852f3f402a7103979b5778a is restarting, wait until the container is running")","type":"danger"}
| SQL upgrade iteration #4
| No valid reponse for mysql_upgrade was received, debug output:
| {"msg":"409 Client Error: Conflict ("Container 8e1cb5c0ec328fe922c23b489e7eb4a6183587385852f3f402a7103979b5778a is restarting, wait until the container is running")","type":"danger"}
| SQL upgrade iteration #5
| No valid reponse for mysql_upgrade was received, debug output:
| {"msg":"409 Client Error: Conflict ("Container 8e1cb5c0ec328fe922c23b489e7eb4a6183587385852f3f402a7103979b5778a is restarting, wait until the container is running")","type":"danger"}
| Tried to upgrade MySQL and failed, giving up after 5 retries and starting container (oops, not good)
| MySQL mysql_tzinfo_to_sql - debug output:
| {"msg":"409 Client Error: Conflict ("Container 8e1cb5c0ec328fe922c23b489e7eb4a6183587385852f3f402a7103979b5778a is restarting, wait until the container is running")","type":"danger"}
| Running DB init...
| Fatal error: Uncaught PDOException: SQLSTATE[HY000] [2002] Connection refused in /web/inc/init_db.inc.php:1175
| Stack trace:
| #0 /web/inc/init_db.inc.php(1175): PDO->__construct('mysql:unix_sock...', 'mailcow', 'McCR4s6Vjybo32Y...', Array)
| #1 {main}
| thrown in /web/inc/init_db.inc.php on line 1175
| Rebuilding domain map in Redis...
| ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)
| ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)
| ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)
| [11-Mar-2020 13:34:25] NOTICE: fpm is running, pid 1
| [11-Mar-2020 13:34:25] NOTICE: ready to handle connections
| 172.22.1.12 - 11/Mar/2020:13:34:39 +0100 "HEAD /settings.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:34:39 +0100 "GET /settings.php" 200
| 172.22.1.12 - 11/Mar/2020:13:34:40 +0100 "HEAD /forwardinghosts.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:34:40 +0100 "GET /forwardinghosts.php" 200
| 172.22.1.12 - 11/Mar/2020:13:35:26 +0100 "HEAD /settings.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:35:26 +0100 "GET /settings.php" 200
| 172.22.1.12 - 11/Mar/2020:13:35:36 +0100 "HEAD /forwardinghosts.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:35:36 +0100 "GET /forwardinghosts.php" 200
| 172.22.1.12 - 11/Mar/2020:13:36:13 +0100 "HEAD /forwardinghosts.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:36:13 +0100 "GET /forwardinghosts.php" 200
| 172.22.1.12 - 11/Mar/2020:13:36:24 +0100 "HEAD /settings.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:36:24 +0100 "GET /settings.php" 200
| 172.22.1.12 - 11/Mar/2020:13:36:57 +0100 "HEAD /settings.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:36:57 +0100 "GET /settings.php" 200
| 172.22.1.12 - 11/Mar/2020:13:37:08 +0100 "HEAD /forwardinghosts.php" 200
| 172.22.1.12 - 11/Mar/2020:13:37:08 +0100 "GET /forwardinghosts.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:37:39 +0100 "HEAD /settings.php" 200
| 172.22.1.12 - 11/Mar/2020:13:37:39 +0100 "GET /settings.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:37:43 +0100 "HEAD /forwardinghosts.php" 200
| 172.22.1.12 - 11/Mar/2020:13:37:43 +0100 "GET /forwardinghosts.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:38:13 +0100 "HEAD /settings.php" 200
| 172.22.1.12 - 11/Mar/2020:13:38:13 +0100 "GET /settings.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:38:42 +0100 "HEAD /forwardinghosts.php" 200
| 172.22.1.12 - 11/Mar/2020:13:38:42 +0100 "GET /forwardinghosts.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:38:55 +0100 "HEAD /settings.php" 200
| 172.22.1.12 - 11/Mar/2020:13:38:55 +0100 "GET /settings.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:39:18 +0100 "HEAD /forwardinghosts.php" 200
| 172.22.1.12 - 11/Mar/2020:13:39:18 +0100 "GET /forwardinghosts.php" 200
| 172.22.1.12 - [email protected] 11/Mar/2020:13:39:34 +0100 "GET /sogo-auth.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:39:54 +0100 "HEAD /settings.php" 200
| 172.22.1.12 - 11/Mar/2020:13:39:54 +0100 "GET /settings.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:39:54 +0100 "HEAD /forwardinghosts.php" 200
| 172.22.1.12 - 11/Mar/2020:13:39:54 +0100 "GET /forwardinghosts.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:40:10 +0100 "GET /index.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:40:14 +0100 "GET /index.php" 200
| 172.22.1.12 - 11/Mar/2020:13:40:15 +0100 "GET /index.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:40:35 +0100 "HEAD /settings.php" 200
| 172.22.1.12 - 11/Mar/2020:13:40:35 +0100 "GET /settings.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:40:52 +0100 "HEAD /forwardinghosts.php" 200
| 172.22.1.12 - 11/Mar/2020:13:40:52 +0100 "GET /forwardinghosts.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:40:58 +0100 "GET /index.php" 200
| 172.22.1.12 - 11/Mar/2020:13:41:00 +0100 "GET /index.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:41:01 +0100 "GET /index.php" 200
| 172.22.1.12 - 11/Mar/2020:13:41:02 +0100 "GET /index.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:41:03 +0100 "GET /index.php" 200
| 172.22.1.12 - 11/Mar/2020:13:41:07 +0100 "GET /index.php" 200
| fd4d:6169:6c63:6f77::10 - 11/Mar/2020:13:41:22 +0100 "HEAD /settings.php" 200
| 172.22.1.12 - 11/Mar/2020:13:41:22 +0100 "GET /settings.php" 200
| [11-Mar-2020 13:41:22] NOTICE: Finishing ...
| [11-Mar-2020 13:41:22] NOTICE: exiting, bye-bye!
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Waiting for SQL...
| Uptime: 1 Threads: 8 Questions: 1 Slow queries: 0 Opens: 17 Flush tables: 1 Open tables: 11 Queries per second avg: 1.000
| We are master, preparing...
| MySQL @ 8e1cb5c0ec328fe922c23b489e7eb4a6183587385852f3f402a7103979b5778a
| SQL upgrade iteration #1
| No valid reponse for mysql_upgrade was received, debug output:
| {"msg":"mysql_upgrade: error running command","text":"OCI runtime exec failed: exec failed: cannot exec a container that has stopped: unknown\r\n","type":"error"}
| SQL upgrade iteration #2
| No valid reponse for mysql_upgrade was received, debug output:
| {"msg":"mysql_upgrade: error running command","text":"Version check failed. Got the following error when calling the 'mysql' command line client\nERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)\nFATAL ERROR: Upgrade failed\n","type":"error"}
| SQL upgrade iteration #3
| No valid reponse for mysql_upgrade was received, debug output:
| {"msg":"mysql_upgrade: error running command","text":"Version check failed. Got the following error when calling the 'mysql' command line client\nERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)\nFATAL ERROR: Upgrade failed\n","type":"error"}
| SQL upgrade iteration #4
| No valid reponse for mysql_upgrade was received, debug output:
| {"msg":"mysql_upgrade: error running command","text":"Version check failed. Got the following error when calling the 'mysql' command line client\nERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)\nFATAL ERROR: Upgrade failed\n","type":"error"}
| SQL upgrade iteration #5
| No valid reponse for mysql_upgrade was received, debug output:
| {"msg":"mysql_upgrade: error running command","text":"Version check failed. Got the following error when calling the 'mysql' command line client\nERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)\nFATAL ERROR: Upgrade failed\n","type":"error"}
| Tried to upgrade MySQL and failed, giving up after 5 retries and starting container (oops, not good)
| MySQL mysql_tzinfo_to_sql - debug output:
| {"msg":"409 Client Error: Conflict ("Container 8e1cb5c0ec328fe922c23b489e7eb4a6183587385852f3f402a7103979b5778a is restarting, wait until the container is running")","type":"danger"}
| Running DB init...
| Fatal error: Uncaught PDOException: SQLSTATE[HY000] [2002] Connection refused in /web/inc/init_db.inc.php:1175
| #0 /web/inc/init_db.inc.php(1175): PDO->__construct('mysql:unix_sock...', 'mailcow', 'McCR4s6Vjybo32Y...', Array)
| #1 {main}
| thrown in /web/inc/init_db.inc.php on line 1175
| Rebuilding domain map in Redis...
| ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)
| ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)
| ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)
| [11-Mar-2020 13:42:26] NOTICE: fpm is running, pid 1
| [11-Mar-2020 13:42:26] NOTICE: ready to handle connections
| 172.22.1.5 - 11/Mar/2020:13:42:37 +0100 "HEAD /forwardinghosts.php" 200
| fd4d:6169:6c63:6f77::5 - 11/Mar/2020:13:42:37 +0100 "GET /forwardinghosts.php" 200
| 172.22.1.5 - 11/Mar/2020:13:42:39 +0100 "HEAD /settings.php" 200
| 172.22.1.5 - 11/Mar/2020:13:42:39 +0100 "GET /settings.php" 200
| fd4d:6169:6c63:6f77::5 - 11/Mar/2020:13:43:15 +0100 "HEAD /forwardinghosts.php" 200
| 172.22.1.5 - 11/Mar/2020:13:43:15 +0100 "GET /forwardinghosts.php" 200
| fd4d:6169:6c63:6f77::5 - 11/Mar/2020:13:43:20 +0100 "HEAD /settings.php" 200
| 172.22.1.5 - 11/Mar/2020:13:43:20 +0100 "GET /settings.php" 200

Reproduction of said bug:
The messages appear after each restart.

System information:
The following message is in the top center of the web page:

Connection to database failed.

The following error was reported:
SQLSTATE[HY000] [2002] Connection refused

andryyy · 2020-03-11T13:00:45Z

Can't replicate it nor have I seen this anywhere.

You should check the whole stack for inconsistency and debug Docker. Don't know about your system, you deleted most of the issue template. :)

It is necessary to check your system. This does not look too good. Try the support channels. I cannot think of a general bug, tbh. What do you think, @mkuron?

mkuron · 2020-03-11T18:50:04Z

Is there anything in the mysql container's log?

roswitina · 2020-03-11T19:13:05Z

docker-compose logs --tail=200 mysql-mailcow

Attaching to mailcowdockerized_mysql-mailcow_1
mysql-mailcow_1 2020-03-11 20:09:39 0 [Note] mysqld (mysqld 10.3.22-MariaDB-1:10.3.22+mariabionic) starting as process 1 ...
mysql-mailcow_1 2020-03-11 20:09:39 0 [Note] InnoDB: Using Linux native AIO
mysql-mailcow_1 2020-03-11 20:09:39 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mysql-mailcow_1 2020-03-11 20:09:39 0 [Note] InnoDB: Uses event mutexes
mysql-mailcow_1 2020-03-11 20:09:39 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
mysql-mailcow_1 2020-03-11 20:09:39 0 [Note] InnoDB: Number of pools: 1
mysql-mailcow_1 2020-03-11 20:09:39 0 [Note] InnoDB: Using SSE2 crc32 instructions
mysql-mailcow_1 2020-03-11 20:09:39 0 [Note] InnoDB: Initializing buffer pool, total size = 8M, instances = 1, chunk size = 8M
mysql-mailcow_1 2020-03-11 20:09:39 0 [Note] InnoDB: Completed initialization of buffer pool
mysql-mailcow_1 2020-03-11 20:09:39 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
mysql-mailcow_1 2020-03-11 20:09:39 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=49790616752
mysql-mailcow_1 2020-03-11 20:09:39 0 [Note] InnoDB: Starting final batch to recover 1 pages from redo log.
mysql-mailcow_1 2020-03-11 20:09:40 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
mysql-mailcow_1 2020-03-11 20:09:40 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
mysql-mailcow_1 2020-03-11 20:09:40 0 [Note] InnoDB: Creating shared tablespace for temporary tables
mysql-mailcow_1 2020-03-11 20:09:40 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
mysql-mailcow_1 2020-03-11 20:09:40 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
mysql-mailcow_1 2020-03-11 20:09:40 0 [Note] InnoDB: Waiting for purge to start
mysql-mailcow_1 2020-03-11 20:09:40 2 [ERROR] [FATAL] InnoDB: InnoDB is trying to free page [page id: space=1227, page number=6] though it is already marked as free in the tablespace! The tablespace free space info is corrupt. You may need to dump your tables and recreate the whole database!Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
mysql-mailcow_1 200311 20:09:40 [ERROR] mysqld got signal 6 ;
mysql-mailcow_1 This could be because you hit a bug. It is also possible that this binary
mysql-mailcow_1 or one of the libraries it was linked against is corrupt, improperly built,
mysql-mailcow_1 or misconfigured. This error can also be caused by malfunctioning hardware.
mysql-mailcow_1
mysql-mailcow_1 To report this bug, see https://mariadb.com/kb/en/reporting-bugs
mysql-mailcow_1
mysql-mailcow_1 We will try our best to scrape up some info that will hopefully help
mysql-mailcow_1 diagnose the problem, but since we have already crashed,
mysql-mailcow_1 something is definitely wrong and this may fail.
mysql-mailcow_1
mysql-mailcow_1 Server version: 10.3.22-MariaDB-1:10.3.22+mariabionic
mysql-mailcow_1 key_buffer_size=0
mysql-mailcow_1 read_buffer_size=131072
mysql-mailcow_1 max_used_connections=0
mysql-mailcow_1 max_threads=352
mysql-mailcow_1 thread_count=4
mysql-mailcow_1 It is possible that mysqld could use up to
mysql-mailcow_1 key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 413409 K bytes of memory
mysql-mailcow_1 Hope that's ok; if not, decrease some variables in the equation.
mysql-mailcow_1
mysql-mailcow_1 Thread pointer: 0x7f2b84000c08
mysql-mailcow_1 Attempting backtrace. You can use the following information to find out
mysql-mailcow_1 where mysqld died. If you see no messages after this, something went
mysql-mailcow_1 terribly wrong...
mysql-mailcow_1 stack_bottom = 0x7f2b9effccf8 thread_stack 0x20000
mysql-mailcow_1 mysqld(my_print_stacktrace+0x2e)[0x555f6da382ee]
mysql-mailcow_1 mysqld(handle_fatal_signal+0x515)[0x555f6d4d0005]
mysql-mailcow_1 /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7f2bc943b890]
mysql-mailcow_1 /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7)[0x7f2bc894fe97]
mysql-mailcow_1 /lib/x86_64-linux-gnu/libc.so.6(abort+0x141)[0x7f2bc8951801]
mysql-mailcow_1 mysqld(+0xa3ea71)[0x555f6d7a4a71]
mysql-mailcow_1 mysqld(+0xafa5fe)[0x555f6d8605fe]
mysql-mailcow_1 mysqld(+0xafeba7)[0x555f6d864ba7]
mysql-mailcow_1 mysqld(+0xa42d1b)[0x555f6d7a8d1b]
mysql-mailcow_1 mysqld(+0xa5e484)[0x555f6d7c4484]
mysql-mailcow_1 mysqld(+0x9c6d85)[0x555f6d72cd85]
mysql-mailcow_1 mysqld(+0x9c7cfb)[0x555f6d72dcfb]
mysql-mailcow_1 mysqld(+0x9830a0)[0x555f6d6e90a0]
mysql-mailcow_1 mysqld(+0x9f28e7)[0x555f6d7588e7]
mysql-mailcow_1 /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db)[0x7f2bc94306db]
mysql-mailcow_1 /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f2bc8a3288f]
mysql-mailcow_1
mysql-mailcow_1 Trying to get some variables.
mysql-mailcow_1 Some pointers may be invalid and cause the dump to abort.
mysql-mailcow_1 Query (0x0):
mysql-mailcow_1 Connection ID (thread ID): 2
mysql-mailcow_1 Status: NOT_KILLED
mysql-mailcow_1
mysql-mailcow_1 Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,2020-03-11 20:09:40 0 [Note] InnoDB: 10.3.22 started; log sequence number 49790620605; transaction id 7812993
mysql-mailcow_1 in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,2020-03-11 20:09:40 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
mysql-mailcow_1 subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,2020-03-11 20:09:40 0 [Note] Recovering after a crash using tc.log
mysql-mailcow_1 2020-03-11 20:09:40 0 [Note] Starting crash recovery...
mysql-mailcow_1 2020-03-11 20:09:40 0 [Note] Crash recovery finished.
mysql-mailcow_1 outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,2020-03-11 20:09:40 0 [Note] Server socket created on IP: '::'.
mysql-mailcow_1 condition_pushdown_for_derived=on,split_materialized=on
mysql-mailcow_1
mysql-mailcow_1 The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
mysql-mailcow_1 information that should help you find out what is causing the crash.
mysql-mailcow_1 Writing a core file...
mysql-mailcow_1 Working directory at /var/lib/mysql
mysql-mailcow_1 Resource Limits:
mysql-mailcow_1 Limit Soft Limit Hard Limit Units
mysql-mailcow_1 Max cpu time unlimited unlimited seconds
mysql-mailcow_1 Max file size unlimited unlimited bytes
mysql-mailcow_1 Max data size unlimited unlimited bytes
mysql-mailcow_1 Max stack size 8388608 unlimited bytes
mysql-mailcow_1 Max core file size unlimited unlimited bytes
mysql-mailcow_1 Max resident set unlimited unlimited bytes
mysql-mailcow_1 Max processes unlimited unlimited processes
mysql-mailcow_1 Max open files 1048576 1048576 files
mysql-mailcow_1 Max locked memory 65536 65536 bytes
mysql-mailcow_1 Max address space unlimited unlimited bytes
mysql-mailcow_1 Max file locks unlimited unlimited locks
mysql-mailcow_1 Max pending signals 31757 31757 signals
mysql-mailcow_1 Max msgqueue size 819200 819200 bytes
mysql-mailcow_1 Max nice priority 0 0
mysql-mailcow_1 Max realtime priority 0 0
mysql-mailcow_1 Max realtime timeout unlimited unlimited us
mysql-mailcow_1 Core pattern: core
mysql-mailcow_1
mysql-mailcow_1 Fatal signal 11 while backtracing
mysql-mailcow_1 2020-03-11 20:09:48+01:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.3.22+mariabionic started.
mysql-mailcow_1 2020-03-11 20:09:48+01:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
mysql-mailcow_1 2020-03-11 20:09:48+01:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.3.22+mariabionic started.
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] mysqld (mysqld 10.3.22-MariaDB-1:10.3.22+mariabionic) starting as process 1 ...
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: Using Linux native AIO
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: Uses event mutexes
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: Number of pools: 1
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: Using SSE2 crc32 instructions
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: Initializing buffer pool, total size = 8M, instances = 1, chunk size = 8M
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: Completed initialization of buffer pool
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=49790616752
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: Starting final batch to recover 1 pages from redo log.
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: Creating shared tablespace for temporary tables
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: Waiting for purge to start
mysql-mailcow_1 2020-03-11 20:09:49 2 [ERROR] [FATAL] InnoDB: InnoDB is trying to free page [page id: space=1227, page number=6] though it is already marked as free in the tablespace! The tablespace free space info is corrupt. You may need to dump your tables and recreate the whole database!Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
mysql-mailcow_1 200311 20:09:49 [ERROR] mysqld got signal 6 ;
mysql-mailcow_1 This could be because you hit a bug. It is also possible that this binary
mysql-mailcow_1 or one of the libraries it was linked against is corrupt, improperly built,
mysql-mailcow_1 or misconfigured. This error can also be caused by malfunctioning hardware.
mysql-mailcow_1
mysql-mailcow_1 To report this bug, see https://mariadb.com/kb/en/reporting-bugs
mysql-mailcow_1
mysql-mailcow_1 We will try our best to scrape up some info that will hopefully help
mysql-mailcow_1 diagnose the problem, but since we have already crashed,
mysql-mailcow_1 something is definitely wrong and this may fail.
mysql-mailcow_1
mysql-mailcow_1 Server version: 10.3.22-MariaDB-1:10.3.22+mariabionic
mysql-mailcow_1 key_buffer_size=0
mysql-mailcow_1 read_buffer_size=131072
mysql-mailcow_1 max_used_connections=0
mysql-mailcow_1 max_threads=352
mysql-mailcow_1 thread_count=4
mysql-mailcow_1 It is possible that mysqld could use up to
mysql-mailcow_1 key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 413409 K bytes of memory
mysql-mailcow_1 Hope that's ok; if not, decrease some variables in the equation.
mysql-mailcow_1
mysql-mailcow_1 Thread pointer: 0x7f3960000c08
mysql-mailcow_1 Attempting backtrace. You can use the following information to find out
mysql-mailcow_1 where mysqld died. If you see no messages after this, something went
mysql-mailcow_1 terribly wrong...
mysql-mailcow_1 stack_bottom = 0x7f396e7fbcf8 thread_stack 0x20000
mysql-mailcow_1 mysqld(my_print_stacktrace+0x2e)[0x564970e0b2ee]
mysql-mailcow_1 mysqld(handle_fatal_signal+0x515)[0x5649708a3005]
mysql-mailcow_1 /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7f3999499890]
mysql-mailcow_1 /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7)[0x7f39989ade97]
mysql-mailcow_1 /lib/x86_64-linux-gnu/libc.so.6(abort+0x141)[0x7f39989af801]
mysql-mailcow_1 mysqld(+0xa3ea71)[0x564970b77a71]
mysql-mailcow_1 mysqld(+0xafa5fe)[0x564970c335fe]
mysql-mailcow_1 mysqld(+0xafeba7)[0x564970c37ba7]
mysql-mailcow_1 mysqld(+0xa42d1b)[0x564970b7bd1b]
mysql-mailcow_1 mysqld(+0xa5e484)[0x564970b97484]
mysql-mailcow_1 mysqld(+0x9c6d85)[0x564970affd85]
mysql-mailcow_1 mysqld(+0x9c7cfb)[0x564970b00cfb]
mysql-mailcow_1 mysqld(+0x9830a0)[0x564970abc0a0]
mysql-mailcow_1 mysqld(+0x9f28e7)[0x564970b2b8e7]
mysql-mailcow_1 /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db)[0x7f399948e6db]
mysql-mailcow_1 /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f3998a9088f]
mysql-mailcow_1
mysql-mailcow_1 Trying to get some variables.
mysql-mailcow_1 Some pointers may be invalid and cause the dump to abort.
mysql-mailcow_1 Query (0x0):
mysql-mailcow_1 Connection ID (thread ID): 2
mysql-mailcow_1 Status: NOT_KILLED
mysql-mailcow_1
mysql-mailcow_1 Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
mysql-mailcow_1
mysql-mailcow_1 The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
mysql-mailcow_1 information that should help you find out what is causing the crash.
mysql-mailcow_1 Writing a core file...
mysql-mailcow_1 Working directory at /var/lib/mysql
mysql-mailcow_1 2020-03-11 20:09:49 0 [Note] InnoDB: 10.3.22 started; log sequence number 49790620605; transaction id 7812993
mysql-mailcow_1 Resource Limits:
mysql-mailcow_1 Limit Soft Limit Hard Limit Units
mysql-mailcow_1 Max cpu time unlimited unlimited seconds
mysql-mailcow_1 Max file size unlimited unlimited bytes
mysql-mailcow_1 Max data size unlimited unlimited bytes
mysql-mailcow_1 Max stack size 8388608 unlimited bytes
mysql-mailcow_1 Max core file size unlimited unlimited bytes
mysql-mailcow_1 Max resident set unlimited unlimited bytes
mysql-mailcow_1 Max processes unlimited unlimited processes
mysql-mailcow_1 Max open files 1048576 1048576 files
mysql-mailcow_1 Max locked memory 65536 65536 bytes
mysql-mailcow_1 Max address space unlimited unlimited bytes
mysql-mailcow_1 Max file locks unlimited unlimited locks
mysql-mailcow_1 Max pending signals 31757 31757 signals
mysql-mailcow_1 Max msgqueue size 819200 819200 bytes
mysql-mailcow_1 Max nice priority 0 0
mysql-mailcow_1 Max realtime priority 0 0
mysql-mailcow_1 Max realtime timeout unlimited unlimited us
mysql-mailcow_1 Core pattern: core
mysql-mailcow_1
mysql-mailcow_1 Fatal signal 11 while backtracing

mkuron · 2020-03-11T19:37:46Z

No idea. I‘ve never seen mysqld segfault upon startup. Are you by chance out of memory and have no swap? Otherwise, my only suggestion is to delete the mysql Docker volume and restore the database from backup.

andryyy · 2020-03-11T19:44:12Z

I'd need to see it on the machine. Cannot advise. If it is business critical, consider support. If not, try to restore from a backup. Or stop mailcow, delete all images (not volumes) and pull them again. Try a mariabackup and boot it outside mailcow in a separated mariadb instance to see if it works.

roswitina · 2020-03-11T19:58:13Z

I have a backup of the MySQL database from yesterday. This seems to have solved the problem. But I don't know why the error was in the database, because I don't know enough about it and I certainly don't play around with it.

As far as I have seen, everything is working again. Thanks for the help.

The BACKUP - Script helped me...

andryyy added the incomplete label Mar 11, 2020

roswitina closed this as completed Mar 11, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SQLSTATE[HY000] [2002] Connection refused #3406

SQLSTATE[HY000] [2002] Connection refused #3406

roswitina commented Mar 11, 2020 •

edited

Loading

andryyy commented Mar 11, 2020

mkuron commented Mar 11, 2020

roswitina commented Mar 11, 2020 •

edited

Loading

mkuron commented Mar 11, 2020

andryyy commented Mar 11, 2020

roswitina commented Mar 11, 2020

SQLSTATE[HY000] [2002] Connection refused #3406

SQLSTATE[HY000] [2002] Connection refused #3406

Comments

roswitina commented Mar 11, 2020 • edited Loading

andryyy commented Mar 11, 2020

mkuron commented Mar 11, 2020

roswitina commented Mar 11, 2020 • edited Loading

mkuron commented Mar 11, 2020

andryyy commented Mar 11, 2020

roswitina commented Mar 11, 2020

roswitina commented Mar 11, 2020 •

edited

Loading

roswitina commented Mar 11, 2020 •

edited

Loading