Search code examples
pythonpandasdockerapple-m1

Pandas on Apple Silicon M1 chip within the Ubuntu container

I have a trouble understanding the issue here: We have

  • Docker image ubuntu:20.04
  • MacBook Pro on M1 chip
  • Old pandas version wheel (legacy system)

Within our Docker image, we use Poetry to manage dependencies:

[tool.poetry.dependencies]
python = ">=3.8,<3.9"

pandas = [
    { platform = "linux", url = "***/pandas-0.24.0-cp38-cp38-linux_x86_64.whl"},
    { platform = "darwin", markers="platform_machine=='x86_64'", url = "***/pandas-0.24.0-cp38-cp38-macosx_10_9_x86_64.whl" },
    { platform = "darwin", markers="platform_machine=='arm64'", url = "***/pandas-0.24.0-cp38-cp38-macosx_12_0_arm64.whl" }
]

But when we try to build this image on the M1 machine, we face the error ERROR: pandas-0.24.0-cp38-cp38-linux_x86_64.whl is not a supported wheel on this platform., but it clearly tries to install ubuntu whl, since it is running within Docker image.

When we build the same image on a Linux machine (in CI/CD), everything works fine.

How can we build an image on M1 chip and on Ubuntu?

Solution

  • M1 is an arm64-based chip, but you try to install pandas for x86_64. I'm not a pythonist, but looks like you need to add a dependency for linux arm64:

    pandas = [
    { platform = "linux", markers="platform_machine=='arm64'" … # <- here
    { platform = "linux", markers="platform_machine=='x86_64'", url = "***/pandas-0.24.0-cp38-cp38-linux_x86_64.whl"},
    { platform = "darwin", markers="platform_machine=='x86_64'", url = "***/pandas-0.24.0-cp38-cp38-macosx_10_9_x86_64.whl" },
    { platform = "darwin", markers="platform_machine=='arm64'", url = "***/pandas-0.24.0-cp38-cp38-macosx_12_0_arm64.whl" }
]