Search code examples
python-3.xdjangopostgresqldocker-composepsycopg2

Why is installing python-psycopg2 on my Docker image not preventing the subsequent "No module named 'psycopg2'" error?


I'm using docker-compose v 1.27.4 on Mac. I have this docker-compose.yml file -- two services, a PostGres DB and a Python/Django app ...

services:

  postgres:
    image: postgres:10.5
    ports:
      - 5105:5432
    environment:
      POSTGRES_DB: directory_data
      POSTGRES_USER: chicommons
      POSTGRES_PASSWORD: password

  web:
    restart: always
    build: ./web
    ports:           # to access the container from outside
      - "8000:8000"
    env_file: .env
    environment:
      DEBUG: 'true'
    command: /usr/local/bin/gunicorn directory.wsgi:application --reload -w 2 -b :8000
    volumes:
    - ./web/:/app
    depends_on:
      - postgres 

This is the Dockerfile used to build the Python/Django container ...

FROM python:3.8-slim
  
RUN apt-get update && apt-get install

RUN apt-get install -y dos2unix
RUN apt-get install -y libpq-dev python-dev
RUN apt-get install -y python-psycopg2
RUN apt-get install -y libmariadb-dev-compat libmariadb-dev
RUN apt-get update \
    && apt-get install -y --no-install-recommends gcc \
    && rm -rf /var/lib/apt/lists/*

RUN python -m pip install --upgrade pip

WORKDIR /my-app/

COPY requirements.txt requirements.txt
COPY entrypoint.sh entrypoint.sh
RUN python -m pip install -r requirements.txt

RUN dos2unix /my-app/entrypoint.sh

ENTRYPOINT ["bash", "/my-app/entrypoint.sh"]

However, when I start my application using "docker-compose up," I get the below errors

web_1       | Traceback (most recent call last):
web_1       |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 25, in <module>
web_1       |     import psycopg2 as Database
web_1       | ModuleNotFoundError: No module named 'psycopg2'
web_1       | 
web_1       | During handling of the above exception, another exception occurred:
web_1       | 
web_1       | Traceback (most recent call last):
web_1       |   File "manage.py", line 21, in <module>
web_1       |     main()
web_1       |   File "manage.py", line 17, in main
web_1       |     execute_from_command_line(sys.argv)
web_1       |   File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
web_1       |     utility.execute()
web_1       |   File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 377, in execute
web_1       |     django.setup()
web_1       |   File "/usr/local/lib/python3.8/site-packages/django/__init__.py", line 24, in setup
web_1       |     apps.populate(settings.INSTALLED_APPS)
web_1       |   File "/usr/local/lib/python3.8/site-packages/django/apps/registry.py", line 114, in populate
web_1       |     app_config.import_models()
web_1       |   File "/usr/local/lib/python3.8/site-packages/django/apps/config.py", line 211, in import_models
web_1       |     self.models_module = import_module(models_module_name)
web_1       |   File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
web_1       |     return _bootstrap._gcd_import(name[level:], package, level)
web_1       |   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
web_1       |   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
web_1       |   File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
web_1       |   File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
web_1       |   File "<frozen importlib._bootstrap_external>", line 783, in exec_module
web_1       |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
web_1       |   File "/usr/local/lib/python3.8/site-packages/django/contrib/auth/models.py", line 2, in <module>
web_1       |     from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
web_1       |   File "/usr/local/lib/python3.8/site-packages/django/contrib/auth/base_user.py", line 47, in <module>
web_1       |     class AbstractBaseUser(models.Model):
web_1       |   File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 121, in __new__
web_1       |     new_class.add_to_class('_meta', Options(meta, app_label))
web_1       |   File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 325, in add_to_class
web_1       |     value.contribute_to_class(cls, name)
web_1       |   File "/usr/local/lib/python3.8/site-packages/django/db/models/options.py", line 208, in contribute_to_class
web_1       |     self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
web_1       |   File "/usr/local/lib/python3.8/site-packages/django/db/__init__.py", line 28, in __getattr__
web_1       |     return getattr(connections[DEFAULT_DB_ALIAS], item)
web_1       |   File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 207, in __getitem__
web_1       |     backend = load_backend(db['ENGINE'])
web_1       |   File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 111, in load_backend
web_1       |     return import_module('%s.base' % backend_name)
web_1       |   File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
web_1       |     return _bootstrap._gcd_import(name[level:], package, level)
web_1       |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 29, in <module>
web_1       |     raise ImproperlyConfigured("Error loading psycopg2 module: %s" % e)
web_1       | django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 module: No module named 'psycopg2'

I don't understand why I'm seeing this error especially given the fact I have

RUN apt-get install -y python-psycopg2

in my container. What else do I need to install the psycopg2 library?


Solution

  • I don't think you even need to have RUN apt-get install -y python-psycopg2. You already have RUN apt-get install -y libpq-dev.

    I think you forgot to put psycopg2==2.8.6 inside your requirements.txt file. Your Dockerfile could change to:

    FROM python:3.8-slim
      
    RUN apt-get update && apt-get install
    
    RUN apt-get install -y \
      dos2unix \
      libpq-dev \
      libmariadb-dev-compat \
      libmariadb-dev \
      gcc \
      && apt-get clean
    
    RUN python -m pip install --upgrade pip
    
    WORKDIR /my-app/
    
    COPY requirements.txt requirements.txt
    COPY entrypoint.sh entrypoint.sh
    RUN python -m pip install -r requirements.txt
    
    RUN dos2unix /my-app/entrypoint.sh
    
    ENTRYPOINT ["bash", "/my-app/entrypoint.sh"]
    

    And make sure you have this in requirements.txt

    psycopg2==2.8.6
    

    Btw, you are using both MariaDB and PostgreSQL in the same project?