Search code examples
azuredatatablecommanddatabricksazure-databricks

Delete tables in Databricks

I have the list of five tables name. I need to delete rest of all the tables in data bricks, which is not in the list. I don't know what command or method to be used to solve this. Please help me on this.

Regards, Manoranjini Muthuraj

Solution 
  • #pyspark code
#list of tables to keep
keep_tables = ['table_1', 'table_2', 'table_3', 'table_4', 'table_5']
#get list of all tables from my_database
df = spark.sql('show tables in my_database')
#loop thru the tables and if table not in keep_tables then do the operation on each table (drop/delete/count etc). 
#**Careful** the code will drop tables not in the keep list
for t in df.collect():
    if t not in keep_tables:
        #do the table operation (drop/delete/count etc)
        print('operate on table {}'.format(t.tableName))
        spark.sql('drop table my_database.{}'.format(t.tableName)))