Search code examples
apache-sparkamazon-s3aws-gluedelta-lake

Can AWS Glue crawl Delta Lake table data?


According to the article by Databricks, it is possible to integrate delta lake with AWS Glue. However, I am not sure if it is possible to do it also outside of Databricks platform. Has someone done that? Also, is it possible to add Delta Lake related metadata using Glue crawlers?


Solution

  • It is finally possible to use AWS Glue Crawlers to detect and catalog Delta Tables.

    Here is a blog post explaining how to do it.