is there a way to check if the file was processed by autoloader already?

Question

I upload files to data bricks file system every two day, i want to know if there's a place or log to see if some file is processed or not?

Thanks

Answer 1

Please refer this link:https://docs.databricks.com/ingestion/auto-loader/production.html#querying-files-discovered-by-auto-loader

For python code refer this link:Get the list of loaded files from Databricks Autoloader

sample code SELECT * FROM cloud_files_state('path/to/checkpoint');