Azure pipeline foreach loop

Question

I am having some problem to figure out the issue in this pipeline.

I have a GitHub repo which contains one folder named dist. This folder has inside multiple files and folder. Here is the structure of my git hub:

├── README.md
├── client_pipeline.yaml
├── client_template.yaml
└── dist
    ├── ReportPhishingOutlookAddIn.xml
    └── ReportPhishingOutlookAddInWeb
        ├── Content
        ├── Dialog.html
        ├── Functions
        ├── Images
        └── Scripts

What I want to do, is when I create a new folder in this repo my pipeline should trigger and upload the dist folder content inside this new folder and save it in azure storage.

So I decided to go with a azure pipeline template first to define the logic of the pipeline:

Client_template.yaml

parameters:
  storageaccount: ''
  client: []
steps:
  - ${{ each client in parameters.client }}:
    - task: AzureCLI@2
      displayName: 'Azure CLI'
      inputs:
        AzureSubscription: '<mysubscription>'
        scriptType: bash
        scriptLocation: inlineScript
        inlineScript: |

         az storage blob upload-batch --source dist --destination '$web/${client}' --account-name ${{ parameters.storageaccount}} --output table --no-progress

And I created another pipeline to call this template.

client_pipeline.yaml

variables:
  - name: storageaccount
    value: <mystorageaccount>

steps:
  - template: client_template.yaml
    parameters:
      storageaccount: ${{ variables.storageaccount }}
      client: ["folder","folder1"]

The second pipeline I am using it to give me freedom to add as many clients as I want and the pipeline should just create them in an azure storage account.

When I run my pipeline, it builds everything without any problem, but when I go in my storage account I have 2 issues.

The first one is that the folder name is ${client} and not the actual name of the name I specified in my array.

The second issue is that the upload, uploads only the main folder and none of the subfolders.

Any help guys please?

EDIT: I solved the issue number 1, now I am able to create multiple folders. But still it doesn't copy all the subfolders and file in folder inside dist

Answer 1

By reference to this thread: https://github.com/Azure/azure-cli/issues/5420#issuecomment-487169413,

az storage azcopy blob upload
-c containerName
--account-name accountName
-s directory/path
-d upload/path

If the sub folder is need to be uploaded, the flag --recursive can be added.

Therefore, you could have a recursive flag that when set will upload subdirectories. Including the --recursive flag with az storage blob upload-batch would upload subdirectories. An example could be az storage blob upload-batch -d <container name> -s <path/to/directory> --recursive. Not including the recursive flag would then only upload files of a particular directory.

Update>> As Nayden has resolved this issue by using the upload batch. If the subfolder is empty, it get ignored and not copied. As soon as We put some files inside all the subfolder, the upload-batch made a recursive copy without any extra flag.

Posting here so other community members who get the same issues can find the answer quickly.