Search code examples
azureazure-storageazure-machine-learning-service

Automatically delete files in storage

So I've noticed that whenever I do a machine learning train/retrain (from here), it generates a lot of files in my Azure blob storage as shown here

Screenshot

I wanted to ask if it was possible to automatically delete all these files or prevent them from ever being generated?

Solution

  • For automatically delete all these files in blob storage, you can use the Lifecycle Management of blob storage.

    It's easy to set up a rule and filter, after the rule is set up, all the files will be deleted as per the rule you defined.

    Simple steps:

    1.Nav to azure portal -> your storage account -> Blob services -> Lifecycle Management, then click "Add rule".

    enter image description here

    2.In the "Action set" tab, select Delete blob and fill in the textbox; Then in "Filter set" tab, select a path.

    enter image description here

    For more details/instructions, please follow this article.

    Also note that the rule runs once per day, and for the first time, it may take 24 hours to take effect.