Search code examples
pythonpython-3.xazureazure-blob-storageazure-storage

Set time to live in Azure Blob containers created using BlobServiceClient


I'm currently using the following setup to create containers in an Azure Storage Account, and writing blobs to those containers:

from azure.storage.blob import BlobServiceClient

connstr = "..."
bsc = BlobServiceClient.from_connection_string(connstr)
container_client = bsc.create_container(name="some_container")
blob_client = container_client.upload_blob("some_blob", data="data_item", metadata={})

but nowhere in this flow can I find a way to set a time to live (TTL, or maximum lifecycle time) for these blobs or containers.

From what I've understood you can create rules for the storage account using blob storage lifecycle management rules, but this would complicate the script significantly. I'd ideally like to be able to set a different blob-TTL for each container. Am I missing something here?


Solution

  • I ended up going with a combination of the suggestion from @dasari-kamali and other sources. My solution was that I defined a number of standardized rules in the Lifecycle-Management section of my Storage Account:

    enter image description here

    Here I have specific rules for 1 day, 7 days, etc. Each of the specific rules have key/value filters which determine which blobs they apply to. Here for example is the filter value for "specificTtl3":

    enter image description here

    Then you can use the blob tag values (TTL-in-days=7) to determine which rule they should follow. I additionally have a global blob deletion rule which defines a max for all blobs (without tag filtering).

    This workaround isn't perfect as you need to have predetermined rules in place (meaning you can't dynamically set them to a value that isn't configured), but it works nicely for my application. This way you also avoid having a timed function run at regular intervals.