Search code examples
pythondatabricksazure-databricksdatabricks-sql

Obtain query_id from a query executed in Databricks using the Databricks SQL connector for Python

I am using the Databricks SQL connector for Python to execute queries (cursor.execute(query)) and would like to obtain the execution time for each query. I have already explored the option of retrieving it using the "list queries" endpoint of the Databricks API, which returns a list of queries with their respective IDs and execution times.

However, I am facing difficulty in finding a way to obtain the query_id of the executed query through a cursor method or parameter. I have searched the documentation thoroughly, but haven't been able to find a solution. Could someone please suggest a way to obtain the query_id or execution time through a cursor method or any other parameter? If that is not possible, I would appreciate any other less straightforward solution.

Solution

  • We can obtain the query ID using the active_op_handle object in the cursor.

    Code:

    connection = sql.connect(
  server_hostname=host,
  http_path=http_path,
  access_token=access_token)

cursor = connection.cursor()
cursor.execute('SELECT * FROM RANGE(10)')

if cursor.active_op_handle is not None:
    query_id = UUID(bytes=cursor.active_op_handle.operationId.guid)
    print(query_id)

    Output:

    Enter image description here