I'm working on a actions workflow and wouldlike to download a file from s3. Our s3 is via IBM cloud and seems to have some nuances around usage e.g. one must add --endpoint-url {endpoint}
in between the command to call aws client and the corresponding aws command, e.g. aws --endpoint-url {endpoint} s3 ls
Here are the relevant steps in my workflow.yml file:
jobs:
retreive-model-s3:
runs-on: ubuntu-latest
steps:
- name: checkout current repo
uses: actions/checkout@master
- name: make dir to sync with s3
run: mkdir models
- name: s3
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
AWS_S3_ENDPOINT: ${{ secrets.AWS_S3_ENDPOINT }}
AWS_REGION: 'us-south'
run: |
aws --endpoint-url ${{ secrets.AWS_S3_ENDPOINT }} s3 ls ${{ secrets.AWS_S3_BUCKET }}
aws --endpoint-url ${{ secrets.AWS_S3_ENDPOINT }} s3 cp ${{ secrets.AWS_S3_BUCKET }}/regression_model_full.rds models/regression_model_full.rds
- name: dir after
Up to here works:
aws --endpoint-url ${{ secrets.AWS_S3_ENDPOINT }} s3 ls ${{ secrets.AWS_S3_BUCKET }}
In the actions console output I am able to see the files in ${{ secrets.AWS_S3_BUCKET }}
, including a file called regression_model_full.rds
.
In the next line I attempt to download it to the runner with:
aws --endpoint-url ${{ secrets.AWS_S3_ENDPOINT }} s3 cp ${{ secrets.AWS_S3_BUCKET }}/regression_model_full.rds models/regression_model_full.rds
But this results in error:
usage: aws s3 cp <LocalPath> <S3Uri> or <S3Uri> <LocalPath> or <S3Uri> <S3Uri>
Error: Invalid argument type
Error: Process completed with exit code 252.
Given I am able to successfully run the ls
command for s3 I'm reassured that my connection to s3 is sound. But why would I not be able to download using cp? Is my syntax incorrect?
Plesae check addid s3: ''' aws --endpoint-url ${{ secrets.AWS_S3_ENDPOINT }} s3 cp s3://${{ secrets.AWS_S3_BUCKET }}/regression_model_full.rds models/regression_model_full.rds '''