I have 100 small files in Azure DevOps Repo under folder "Files". Files includes configuration related information. I would like to to upload all files to Azure DevOps Pipeline by Terraform on Azure DevOps Pipeline. I would then process files with ADF and insert data to database.
How can I upload files to Storage? (Files in Repo->Pipeline->Terraform->Azure Storage Blob)
Have you considered Azure File Copy?
Microsoft Doc → https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-file-copy?view=azure-devops
GitHub repo → https://github.com/microsoft/azure-pipelines-tasks/blob/master/Tasks/AzureFileCopyV1/README.md
The task provides the ability to copy files to an Azure blob or directly to Azure VMs. Even when the target is Azure VMs, Azure blobs are used as an intermediary and the files are copied to it first and then downloaded to the VMs. The tasks uses AzCopy, the command-line utility built for fast copying of data from and into Azure storage accounts.