Search code examples
dataformgoogle-workflows

Using an input variable in a list field in a GCP Workflow


I'm trying to set the includedTags via a variable when creating a GCP Dataform Workflow Invocation via GCP Workflows. My relevant GCP Workflows step is:

main:
    params: [args]
    steps:
    - createWorkflowInvocation:
        try:
            call: http.post
            args:
                url: https://dataform.googleapis.com/v1beta1/projects/<endpoint>
                auth:
                    type: OAuth2
                body:
                    compilationResult: <cr>
                    invocationConfig:
                        includedTags: [args.DATAFORM_TAGS]

where args.DATAFORM_TAGS is a comma-separated string such as alerting,daily. This is based on the Dataform docs (although notice that the docs put each tag on a separate line).

This fails with the error:

{"error":{"code":400,"message":"At least one action must be selected for execution.","status":"INVALID_ARGUMENT"}}

args.DATAFORM_TAGS isn't being substituted properly. I've confirmed that the variable is being correctly set (I can print it out in a different step).

I think that the problem is with args.DATAFORM_TAGS being within square brackets. If I replace that line with includedTags: [alerting,daily] then it works. I've tried various other things such as:

  • includedTags: [${args.DATAFORM_TAGS}]
  • includedTags: ${"[" + args.DATAFORM_TAGS + "]"}
  • includedTags: text.split(args.DATAFORM_TAGS, ",")
  • includedTags: text.split("alerting,daily", ",")
  • sending the body as a JSON message rather than letting GCP Workflows convert directly

but none work.


Solution

  • includedTags: ${text.split(args.DATAFORM_TAGS, ",")} works. It's not far from some other solutions that I'd tried, the key thing that I was missing is the ${} wrapping around it.

    The full working code:

    main:
        params: [args]
        steps:
        - createWorkflowInvocation:
            try:
                call: http.post
                args:
                    url: https://dataform.googleapis.com/v1beta1/projects/<endpoint>
                    auth:
                        type: OAuth2
                    body:
                        compilationResult: <cr>
                        invocationConfig:
                            includedTags: ${text.split(args.DATAFORM_TAGS, ",")}