Search code examples
airflow

Trying to download apache airflow with pip but error pops up when "building wheels for collected packages: google-re2"

I'm on MacOS Mojave, Python 3.8.3, and pip 23.2. I made a py_env, activated it, and ran

pip install apache-airflow==2.6.3' \
 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.6.3/constraints-3.8.txt".

At first I had an error saying it couldn't find file "pybind11". I installed pybind11 by running python -m pip install pybind11, but now theres multiple unknown type errors and other errors.

python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [140 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.9-x86_64-cpython-38
copying re2.py -> build/lib.macosx-10.9-x86_64-cpython-38
running build_ext
building '_re2' extension
creating build/temp.macosx-10.9-x86_64-cpython-38
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -
DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -
I/Users/chu/opt/anaconda3/include -arch x86_64 -
I/Users/chu/opt/anaconda3/include -arch x86_64 -
I/Users/chu/Desktop/Airflow_test/py_env/lib/python3.8/site-
packages/pybind11/include -
I/Users/chu/Desktop/Airflow_test/py_env/include -
I/Users/chu/opt/anaconda3/include/python3.8 -c _re2.cc -o
build/temp.macosx-10.9-x86_64-cpython-38/_re2.o -fvisibility=hidden
In file included from _re2.cc:11:
...
/Users/chu/Desktop/Airflow_test/py_env/lib/python3.8/site-
packages/pybind11/include/pybind11/detail/common.h:688:18: warning: 
alias declarations are a C++11 extension [-Wc++11-extensions]
using type = ISeq;
^
fatal error: too many errors emitted, stopping now [-ferror-
limit=]
18 warnings and 20 errors generated.
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a 
problem with pip.
ERROR: Failed building wheel for google-re2
Running setup.py clean for google-re2
Failed to build google-re2
ERROR: Could not build wheels for google-re2, which is required to 
install pyproject.toml-based projects
Solution

  • I ran into a similar issue in my MacOS environment and was able to solve it by re-installing python.

    As Ankit Chaurasia pointed out the issue most likely is stemming from your python version having been installed using an older version of MacOS.

    For me, using pyenv, I simply installed a new python version pyenv install 3.9.6 activated it locally pyenv local 3.9.6 and created a new virtual environment with that python version python3.9 -m venv my_venv.

    From there, I followed the install steps again from airflow without any issues.

    pip install 'apache-airflow==2.6.3' \
 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.6.3/constraints-3.9.txt"

    Hope this helps.