Search code examples
pythonamazon-web-servicesaws-lambdapython-imaging-libraryaws-lambda-layers

How to convert JFIF to JPEG on AWS Lambda in Python 3.11? Using Pillow gives difficult error

I've tried to get Pillow to work on AWS Lambda on x86 using Lambda Layers (I built the python dependency zip using a docker ubuntu image on my computer, and I've also tried building it on a docker Amazon Linux image), but I get errors that I don't know how to fix. The errors seem to throw on the line

from PIL import Image

The version of Pillow I've installed via lambda layer is 10.0.1 The error that I get is:

[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': cannot import name '_imaging' from 'PIL' (/opt/python/lib/python3.11/site-packages/PIL/__init__.py)

This error only happens for me on AWS Lambda, not on any of my local machines.

I don't import '_imaging' anywhere directly in my code, so I guess the import is happening from inside of PIL/__init__.py. (Although when I opened that file, I don't see that import statement directly in that file, either.)

I've tried following advice of a number of other stack overflow answers, including the top answer here: Python Layer Image Failing: "Unable to import module 'lambda_function': cannot import name '_imaging' from 'PIL'"

The top answer recommends:

  1. Use an older version of Pillow (pre 2.1.0)
  2. Find where you are importing _imaging and replace it with the updated from PIL.Image import core as _imaging
  3. Update to the current version of Pillow

When I tried using an older version of Pillow (2.0.0), I ended up getting the following error instead:

Unable to import module 'lambda_function': The _imaging C module is not installed"

When I tried looking for where _imaging is getting imported, I was unable to find it (as described above).

And I'm already at the current version of Pillow, so I can't update it.

Please help me figure out what's going wrong! Or if you have an alternative suggestion for a way for me to convert jfif to jpeg, please let me know. I tried imageio, but I'm running into errors on lambda regarding imageio's dependency on numpy.
Thanks!

Solution

  • I ended up solving this in an indirect way. I didn't get the library working on default lambda, but I ended up building a docker container with my code and all my dependencies, and ran it on lambda via ECR. So now my whole code is in a container, not just the layer with dependencies.