Search code examples
pipelineaws-cliamazon-sagemaker

AWS Sagemaker pipeline definition error while running from aws-cli

Im trying to integrate Sagemaker pipeline with Jenkins. Im using aws-cli ( version - 2.1.24 ).

Since this version doesnt support --pipeline-definition-s3-location, Im trying to do something like below -

aws s3 cp s3://some_bucket/folder1/pipeine_definition.json - | \
jq -c . | \
tee /dev/stderr | \
xargs -0 -I{} aws sagemaker update-pipeline --pipeline-name "pipelinename" --role-arn "arn:aws:iam::<account_id>:role/sagemaker-role" --pipeline-definition '{}'

And I found this error - An error occurred (ValidationException) when calling the UpdatePipeline operation: Pipeline definition: At least 1 step must be provided

When I recheck the definition.json, Im able to see the steps defined inside json.

Can someone help me?

I tried adding the quotes for --pipeline-definition, which isnt working.

Since jenkins has aws-cli 2.1.24 version, I want to someone copy the contents of json file in s3 and pass it to --pipeline-definition argument using aws sagemaker --update-pipeline command.

Solution

  • I resolved this using below shell command -

    pipeline_definition_json=$(aws s3 cp "s3://$S3_BUCKET/<folder_1>/sagemaker_pipeine_definition.json" - | tr -d '\n') 

aws sagemaker update-pipeline --pipeline-name $PIPELINE_NAME --role-arn $ROLE_ARN --pipeline-definition "$pipeline_definition_json"