Search code examples
databricksazure-databricksdatabricks-sql

How to capture the number of rows inserted/updated to Delta table in Databricks


I am trying to capture the number of rows inserted/updated for a table in databricks

Please see the below screenshot for more details. I need to pull the number 31 from num_affected_rows

enter image description here


Solution

  • In my case I'm using the DatabricksSqlHook() in Airflow and in order to extract such results:

    from airflow.providers.databricks.hooks.databricks_sql import DatabricksSqlHook
    
    
    db = DatabricksSqlHook(databricks_conn_id='databricks_default')
    
    conn = db.get_conn()
    cursor = conn.cursor()
    query_metadata = cursor.execute(query)
    
    total_rows = query_metadata.fetchone()['num_affected_rows']