Wait until AWS Glue crawler has finished running

Question

In the documentation, I cannot find any way of checking the run status of a crawler. The only way I am doing it currently is constantly checking AWS to check if the file/table has been created.

Is there a better way to block until crawler finishes its run?

Answer 1

You can use boto3 (or similar) to do it. There is the get_crawler method. You will find needed information in "LastCrawl" section

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/glue.html#Glue.Client.get_crawler