Search code examples
amazon-web-servicesamazon-sagemaker

Unable to pull the pre-built Inferentia Pytorch Sagemaker image


I want to extend the Inferentia Pytorch pre-built SageMaker docker container found here but I am not being able to pull the base image...

First I do -

$ aws ecr get-login-password --region eu-west-2 | docker login --username AWS --password-stdin 205493899709.dkr.ecr.eu-west-2.amazonaws.com

Login Succeeded

Then -

$ docker pull 205493899709.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-neo-pytorch:1.9-inf-py3

To which I get the following error:

Error response from daemon: pull access denied for 205493899709.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-neo-pytorch, repository does not exist or may require 'docker login': denied: User: arn:aws:iam::[redacted]:user/[redacted] is not authorized to perform: ecr:BatchGetImage on resource: arn:aws:ecr:eu-west-2:205493899709:repository/sagemaker-neo-pytorch because no resource-based policy allows the ecr:BatchGetImage action

The AWS user already has the AmazonEC2ContainerRegistryPowerUser permission attached, which allows the ecr:BatchGetImage action.

Also, I am able to run docker pull 763104351884.dkr.ecr.eu-west-2.amazonaws.com/pytorch-inference:2.0.0-gpu-py310 the same way without any issues


Solution

  • The ECR repository for this image appears to restrict direct pulling. This limitation is not related to the IAM permissions in your account but is inherent to the target ECR repository (205493899709.dkr.ecr.eu-west-2.amazonaws.com).

    Some SageMaker images are designed to be used as they are and cannot serve as source images in a Dockerfile. These images are typically configured for direct use within the SageMaker environment and cannot be extended using Docker.

    I suggest looking into whether you can bundle your dependencies and requirements directly with your model (see here for more details) and use the image as is. If this isn't feasible, you might need to create a completely new image from scratch.

    You might also consider reaching out to AWS support to discuss your specific requirements.