pipe the output of `aws glue list-jobs` to `aws glue start-job-run`

Question

How do I pipe the output of aws glue list-jobs... to aws glue start-job-run in powershell and bash?

Ex. something like:

aws glue list-jobs --tags name=something (magic here) | aws glue start-job-run (magic here)

This being a solution to the problem: quickly starting multiple jobs using the universal aws cli.

Thank you.

Answer 1

• Make the aws glue list-jobs call output the job names as text (--output text).

• Using % (a built-in alias of the ForEach-Object cmdlet), pass each job name to aws glue start-job-run via the --job-name parameter and the automatic $_ variable.

# % is short for ForEach-Object
aws glue list-jobs --tags name=something --output text | 
  % { aws glue start-job-run --job-name $_ }

Update: You report that you ended up using the following, relying on the default output format, JSON:

# % is short for ForEach-Object
aws glue list-jobs --tags name=something  | 
  ConvertFrom-Json |
  Select-Object -ExpandProperty JobNames |
  % { aws glue start-job-run --job-name $_ }