Search code examples
argo-workflows

How to pass multiple lists in a single loop in argo workflows


I want to pass elements of multiple lists in a single loop using argo templates

I have 3 lists like below

effect = ["Allow", "Allow", "Allow"]
resource = ["*", "*", "*"]
action = ["ec2:CreateTags", "ec2:DescribeInstances", "ec2:AuthorizeSecurityGroupIngress"]

I have a IAM policy which I need to construct with the argo templates. The below IAM policy takes elements from 3 lists in each loop. How can we pass these three lists elements in a single loop?

I referred to argo documentations, there's only withItems/withParams loop is available which takes only one list at a time

I tried the below method but, it is not working

- name: policy
  value: |-
  <<-EOP
  {
    "Effect": "{{item.1}}",
    "Action": "{{item.2}}",
    "Resource": "{{item.3}}"
  }
  EOP
  withTogether:
   - "{{inputs.parameters.effect}}"
   - "{{inputs.parameters.actions}}"
   - "{{inputs.parameters.resources}}"

If it is not supported in argo, is there any alternate way that we can achieve this?


Solution

  • Don't use withItems/withParams for simple JSON manipulation. Argo Workflows represents each iteration of these loops with at least one Pod. That's slow.

    I'd recommend using a familiar programming language and a script template to perform the work.

    Argo has a simple, built-in "expressions tag" templating tool which you could use to perform the same mutation.

    apiVersion: argoproj.io/v1alpha1
    kind: Workflow
    metadata:
      generateName: so-69180308-
    spec:
      arguments:
        parameters:
          - name: effects
            value: '["Allow", "Allow", "Allow"]'
          - name: resources
            value: '["*", "*", "*"]'
          - name: actions
            value: '["ec2:CreateTags", "ec2:DescribeInstances", "ec2:AuthorizeSecurityGroupIngress"]'
      entrypoint: main
      templates:
        - name: main
          script:
            env:
              - name: POLICIES
                value: >-
                  {{=
                    toJson(
                      map(
                        0..(len(jsonpath(workflow.parameters['effects'], '$'))-1),
                        {
                          {
                            effect: jsonpath(workflow.parameters['effects'], '$')[#],
                            resource: jsonpath(workflow.parameters['resources'], '$')[#],
                            action: jsonpath(workflow.parameters['actions'], '$')[#]
                          }
                        }
                      )
                    )
                  }}
            image: debian:9.4
            command: [bash]
            source: |
              echo "$POLICIES"
    

    Output:

    [{"action":"ec2:CreateTags","effect":"Allow","resource":"*"},{"action":"ec2:DescribeInstances","effect":"Allow","resource":"*"},{"action":"ec2:AuthorizeSecurityGroupIngress","effect":"Allow","resource":"*"}]
    

    I'd recommend against going the expression tag route. It's a less well-known language and will be more difficult for others to maintain.