Search code examples
pysparkdatabricks

What schemas does Databricks create automatically?

The Databricks "Data Explorer" exposes schemas from the Spark catalog: https://docs.databricks.com/data/index.html

Certain schemas can be automatically created by Databricks. For example, if you attempt to create a table without specifying a schema, it will be assumed to go to the schema called default.

What are other automatically generated schemas?

Note: While I welcome all input, I would greatly appreciate an answer supported by some source (ideally Databricks documentation).

Solution

  • it's really dependent on different factors:

    • default exists by default when you use Hive Metastore and Unity Catalog (UC)
    • if you use Unity Catalog, then information_schema schema/database is created automaticall - it's created for each catalog in the Unity Catalog metastore
    • the main catalog is also created automatically in each UC metastore
    • the samples catalog is automatically available in the Unity Catalog - it contains sample data
    • there could be also system catalog in the Unity Catalog, but it depends on the configuration, previews, etc.