I am trying to capture the number of rows inserted/updated for a table in databricks
Please see the below screenshot for more details. I need to pull the number 31 from num_affected_rows
In my case I'm using the DatabricksSqlHook()
in Airflow and in order to extract such results:
from airflow.providers.databricks.hooks.databricks_sql import DatabricksSqlHook
db = DatabricksSqlHook(databricks_conn_id='databricks_default')
conn = db.get_conn()
cursor = conn.cursor()
query_metadata = cursor.execute(query)
total_rows = query_metadata.fetchone()['num_affected_rows']