Search code examples
mysqlgalera

Galera clusters with too many connections issue


When I run a command to see our database variables:

show status like "%onn%"

I found that there are way too many connections for 2 people connecting to:

MariaDB [(none)]> show status like "%onn%";
+-----------------------------------------------+-------+
| Variable_name                                 | Value |
+-----------------------------------------------+-------+
| Aborted_connects                              | 0     |
| Connection_errors_accept                      | 0     |
| Connection_errors_internal                    | 0     |
| Connection_errors_max_connections             | 0     |
| Connection_errors_peer_address                | 0     |
| Connection_errors_select                      | 0     |
| Connection_errors_tcpwrap                     | 0     |
| Connections                                   | 56711 |
| Max_used_connections                          | 11    |
| Performance_schema_session_connect_attrs_lost | 0     |
| Slave_connections                             | 0     |
| Slaves_connected                              | 0     |
| Ssl_client_connects                           | 0     |
| Ssl_connect_renegotiates                      | 0     |
| Ssl_finished_connects                         | 0     |
| Threads_connected                             | 5     |
| wsrep_connected                               | ON    |
+-----------------------------------------------+-------+
17 rows in set (0.00 sec)

The number of Connections doesn't go down. It keeps increasing.

CPU Utilization (Percent): 0.82%
KiB Mem :  4045012 total,  1625720 free,   718928 used

How can I diagnosis this issue to find the root cause?

It crashed and I suspect it could crash again.

UPDATED: According to this MySQL show status - active or total connections?, Threads_connected is the value I should look for open connections. Phew! Great!

However, my processlist could be very suspicious. Notice the Time column? Are they supposed to be that high?

MariaDB [(none)]> SHOW FULL processlist;
+------+---------------+-----------------+------------+---------+-------+-------------------------+-----------------------+----------+
| Id   | User          | Host            | db         | Command | Time  | State                   | Info                  | Progress |
+------+---------------+-----------------+------------+---------+-------+-------------------------+-----------------------+----------+
|    1 | system user   |                 | NULL       | Sleep   | 23321 | wsrep aborter idle      | NULL                  |    0.000 |
|    2 | system user   |                 | NULL       | Sleep   |  3955 | committed 3040010       | NULL                  |    0.000 |
|    5 | system user   |                 | NULL       | Daemon  |  NULL |                         | NULL                  |    0.000 |
|    6 | system user   |                 | NULL       | Daemon  |  NULL |                         | NULL                  |    0.000 |
|    3 | system user   |                 | NULL       | Daemon  |  NULL |                         | NULL                  |    0.000 |
|    4 | system user   |                 | NULL       | Daemon  |  NULL |                         | NULL                  |    0.000 |
|    7 | system user   |                 | NULL       | Daemon  |  NULL | InnoDB shutdown handler | NULL                  |    0.000 |
|    9 | system user   |                 | NULL       | Sleep   |  3957 | committed 3040009       | NULL                  |    0.000 |
|   10 | system user   |                 | NULL       | Sleep   |  3950 | committed 3040033       | NULL                  |    0.000 |
|   11 | system user   |                 | NULL       | Sleep   |  3954 | committed 3040022       | NULL                  |    0.000 |
|   12 | system user   |                 | NULL       | Sleep   |  3954 | committed 3040020       | NULL                  |    0.000 |
|   16 | system user   |                 | NULL       | Sleep   |  3953 | committed 3040030       | NULL                  |    0.000 |
|   17 | system user   |                 | NULL       | Sleep   |  3949 | committed 3040034       | NULL                  |    0.000 |
|   15 | system user   |                 | NULL       | Sleep   |  3953 | committed 3040031       | NULL                  |    0.000 |
| 9670 | xxx           | localhost:51574 | tabl_name  | Sleep   |     4 |                         | NULL                  |    0.000 |
| 9671 | xxx           | localhost:51576 | NULL       | Sleep   |    53 |                         | NULL                  |    0.000 |
| 9939 | xxx           | localhost       | NULL       | Sleep   |   123 |                         | NULL                  |    0.000 |
| 9942 | xxx           | localhost       | NULL       | Query   |     0 | init                    | SHOW FULL processlist |    0.000 |
+------+---------------+-----------------+------------+---------+-------+-------------------------+-----------------------+----------+
18 rows in set (0.00 sec)

Solution

  • Unpanic. Connections is a counter. These are more interesting Connections / Uptime and Threads_running. Note that Max_used_connections is only 11. That's OK.

    Please provide * Ram size * SHOW VARIABLES; * SHOW GLOBAL STATUS; -- after it has been running at least a day

    Do you have some form of "connection pooling"? Proxy server? MaxScale? HAProxy? Perhaps you are seeing "pooled" connections, not real ones.