Search code examples
python-3.xloggingtimeoutpymysql

pymysql cannot execute query remotely


I'm executing a set of very heavy queries remotely to the company databases from our central server. Unfortunately, python logging raises this error from the database when trying to execute certain INSERT query.

/usr/local/lib/python3.5/dist-packages/pymysql/cursors.py:329: Warning: (1592, 'Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statement is unsafe because it uses a UDF which may not return the same value on the slave.')
  self._do_get_result()
/usr/local/lib/python3.5/dist-packages/pymysql/cursors.py:329: Warning: (1592, 'Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statements writing to a table with an auto-increment column after selecting from another table are unsafe because the order in which rows are retrieved determines what (if any) rows will be written. This order cannot be predicted and may differ on master and the slave.')
  self._do_get_result()
/usr/local/lib/python3.5/dist-packages/pymysql/cursors.py:329: Warning: (1592, 'Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statement is unsafe because it invokes a trigger or a stored function that inserts into an AUTO_INCREMENT column. Inserted values cannot be logged correctly.')
  self._do_get_result()

wihout python logging, the error es expressed differently, and does not locks the python program execution:

[2019-09-27 13:24:47,228 root ERROR] Fallo ejecucion de query: INSERT INTO movimiento (numero_sala, tipo_movimiento, fec_movimiento, hora_movimiento, nro_tarjeta, id_maquina, monto, lugar_im, fecha_hora_im)                     VALUES ( 3, 20, current_date, current_time, 157299522, 0, 40.000000, 2, current_timestamp)
Traceback (most recent call last):
  File "/opt/cruciscripts/crucidmcs/connectionManager.py", line 58, in alter
    cursor.execute(query)
  File "/usr/local/lib/python3.5/dist-packages/pymysql/cursors.py", line 170, in execute
    result = self._query(query)
  File "/usr/local/lib/python3.5/dist-packages/pymysql/cursors.py", line 328, in _query
    conn.query(q)
  File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 517, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 732, in _read_query_result
    result.read()
  File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1075, in read
    first_packet = self.connection._read_packet()
  File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 684, in _read_packet
    packet.check_error()
  File "/usr/local/lib/python3.5/dist-packages/pymysql/protocol.py", line 220, in check_error
    err.raise_mysql_exception(self._data)
  File "/usr/local/lib/python3.5/dist-packages/pymysql/err.py", line 109, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.InternalError: (1205, 'Lock wait timeout exceeded; try restarting transaction')

Solution

  • Depending on the SQL engine, there are limitations for the size of a query. The obvious solution was to partition the query in smaller chunks.