Search code examples
blobazure-data-factory

Is there a way we can filter blobs that are not in archive tier in Azure Blob?

I have a blob container say "demo". It has many files, few are in hot tier and few are in archive tier. I want to process only those files which are in Hot ignoring archive tier files.

  1. using getmetadata activity lists all files including archive tier files
  2. using "az storage blob list throws" error - 'This operation is not permitted on an archived blob.'

Pl direct me in the right direction.

Solution

  • You can use List Blobs operation which returns a list of the blobs under the specified container.

    Method: GET

    Request URI: https://myaccount.blob.core.windows.net/mycontainer?restype=container&comp=list

    HTTP Version: HTTP/1.1

    This will return the response body in XML format which you can later filter on the basis of access tier names.

    In the above URI, you also need to provide parameter include={versions=2019-12-12} if it doesn't use newest version automatically.

    For version 2017-04-17 and above, List Blobs returns the AccessTier element if an access tier has been explicitly set. For Blob Storage or General Purpose v2 accounts, valid values are Hot/Cool/Archive. If the blob is in rehydrate pending state then ArchiveStatus element is returned with one of the valid values rehydrate-pending-to-hot/rehydrate-pending-to-cool.

    Refer List Blobs for more details.