I am trying to create a AWS Lambda function using Docker Image as source. I am executing the following code as part of the image build phase to download all the dependencies
import logging
logging.basicConfig(level=logging.DEBUG)
from langchain.embeddings import HuggingFaceInstructEmbeddings
from transformers import AutoModelForSequenceClassification, AutoTokenizer
instructor_embeddings = HuggingFaceInstructEmbeddings(model_name="hkunlp/instructor-large",
model_kwargs={"device": "cpu"},encode_kwargs={"batch_size": 1})
# Replace 'model_id' with your specific model identifier
model_id = "SamLowe/roberta-base-go_emotions"
# Download the model and tokenizer
model = AutoModelForSequenceClassification.from_pretrained(model_id,force_download=True)
tokenizer = AutoTokenizer.from_pretrained(model_id,force_download=True)
# Save the model and tokenizer to a directory (e.g., "./models/")
model.save_pretrained("./model")
tokenizer.save_pretrained("./tokenizer")
Here is the dockerfile
# lambda base image for Docker from AWS
FROM public.ecr.aws/lambda/python:latest
# Update the package list and install development tools for C++ support
RUN yum update -y && \
yum install deltarpm -y && \
yum groupinstall "Development Tools" -y
# Export CXXFLAGS environment variable for C++11 support
ENV CXXFLAGS="-std=c++11"
ENV AWS_DEFAULT_REGION="eu-central-1"
ENV TRANSFORMERS_OFFLINE="1"
ENV SENTENCE_TRANSFORMERS_HOME="/root/.cache/huggingface/"
# Install packages
COPY requirements.txt ./
RUN python3 -m pip install -r requirements.txt
# Setup directories
RUN mkdir -p /tmp/content/pickles/ ~/.cached
RUN chmod 0700 ~/.cached
# copy all code and lambda handler
COPY *.py ./
RUN python3 dependency.py
# run lambda handler
CMD ["function.handler"]
While runnging the docker build command, the phase for executing the file dependency.py leads to the below error
docker build . -t ml-server
[+] Building 6.5s (13/13) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.38kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 32B 0.0s
=> [internal] load metadata for public.ecr.aws/lambda/python:latest 1.3s
=> [internal] load build context 0.0s
=> => transferring context: 287B 0.0s
=> [1/9] FROM public.ecr.aws/lambda/python:latest@sha256:d8a8324834a079dbdfc6551831325113512a147bf70003622412565f216 0.0s
=> CACHED [2/9] RUN yum update -y && yum install deltarpm -y && yum groupinstall "Development Tools" -y 0.0s
=> CACHED [3/9] COPY requirements.txt ./ 0.0s
=> CACHED [4/9] RUN echo `whoami` 0.0s
=> CACHED [5/9] RUN python3 -m pip install -r requirements.txt 0.0s
=> [6/9] RUN mkdir -p /tmp/content/pickles/ ~/.cached 0.3s
=> [7/9] RUN chmod 0700 ~/.cached 0.2s
=> [8/9] COPY *.py ./ 0.0s
=> ERROR [9/9] RUN python3 dependency.py 4.6s
------
> [9/9] RUN python3 dependency.py:
#13 1.260 INFO:numexpr.utils:NumExpr defaulting to 4 threads.
#13 2.806 INFO:sentence_transformers.SentenceTransformer:Load pretrained SentenceTransformer: hkunlp/instructor-large
#13 2.807 DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): huggingface.co:443
#13 3.658 DEBUG:urllib3.connectionpool:https://huggingface.co:443 "GET /api/models/hkunlp/instructor-large HTTP/1.1" 200 140783
#13 4.016 Traceback (most recent call last):
#13 4.016 File "/var/task/dependency.py", line 7, in <module>
#13 4.016 instructor_embeddings = HuggingFaceInstructEmbeddings(model_name="hkunlp/instructor-large",
#13 4.016 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#13 4.016 File "/var/lang/lib/python3.11/site-packages/langchain/embeddings/huggingface.py", line 150, in __init__
#13 4.016 self.client = INSTRUCTOR(
#13 4.016 ^^^^^^^^^^^
#13 4.016 File "/var/lang/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 87, in __init__
#13 4.017 snapshot_download(model_name_or_path,
#13 4.017 File "/var/lang/lib/python3.11/site-packages/sentence_transformers/util.py", line 491, in snapshot_download
#13 4.017 path = cached_download(**cached_download_args)
#13 4.017 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#13 4.017 File "/var/lang/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
#13 4.017 return fn(*args, **kwargs)
#13 4.017 ^^^^^^^^^^^^^^^^^^^
#13 4.017 File "/var/lang/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 749, in cached_download
#13 4.018 raise LocalEntryNotFoundError(
#13 4.018 huggingface_hub.utils._errors.LocalEntryNotFoundError: Connection error, and we cannot find the requested files in the cached path. Please try again or make sure your Internet connection is on.
------
executor failed running [/bin/sh -c python3 dependency.py]: exit code: 1
When I try to execute the same file on macOS it runs without an issue. I am able to get packages from pip within the image so network connection should not be an issue
Judging from the error message I thought this would be an error related to the embeddings method, so I add the force_download download flag and set the download batch_size to 1.
That also resulted in the same error The size of the downloaded dependencies is not larger than 3 GB. Is there a CPU bottleneck while running the download pass?
So the issue was because the Environment Variable was set to use Transformers in offline mode
ENV TRANSFORMERS_OFFLINE="1"