Environment properties are not passed to application in Elastic Beanstalk

Question

When deploying my Django project, database settings are not configured because 'RDS_HOSTNAME' in os.environ returns false. In fact no environment properties are available at the time of deployment. All these properties are available after the deployment.

Running /opt/elasticbeanstalk/bin/get-config environment returns following:

{"DJANGO_SETTINGS_MODULE":"myApp.settings","PYTHONPATH":"/var/app/venv/staging-LQM1lest/bin:$PYTHONPATH","RDS_DB_NAME":"ebdb","RDS_HOSTNAME":"xxxx.amazonaws.com","RDS_PASSWORD":"xxxx","RDS_PORT":"xxxx","RDS_USERNAME":"xxxx"}

All RDS prefixed properties are set but still somehow os.environ is unable to read it.

setting.py file:

# [...]

if 'RDS_HOSTNAME' in os.environ:
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.mysql',
            'NAME': os.environ['RDS_DB_NAME'],
            'USER': os.environ['RDS_USERNAME'],
            'PASSWORD': os.environ['RDS_PASSWORD'],
            'HOST': os.environ['RDS_HOSTNAME'],
            'PORT': os.environ['RDS_PORT'],
        }
    }

# [...]

Do I have to make any changes to make these properties available at the time of deployment?

Answer 1

Seems like this is a serious bug and AWS doesn't care about it. There are few ways I came up with to make this work but all of them require logging into the EB environment and do some manual work.

Solution 1

As suggested in comment by hephalump

1. Create an AWS secret manager

2. Check IAM instance profile in EB's environment Configuration->Security->Edit.

3. Then go to IAM user console and go to Roles. From there you can attach policy to the instance profile for secret manager.

4. Once it's done, deploy the project

5. Then login to the environment (eb ssh environment_name).

6. Go to /var/app/current/ directory and run this command: source /var/app/venv/*/bin/activate.

7. Finally run python3 manage.py migrate.

Solution 2

1. Edit .bash_profile and add export these variables at the end of the file:

    export RDS_DB_NAME=your_dbname
    export RDS_USERNAME=user
    export RDS_PASSWORD=pass
    export RDS_HOSTNAME=host_endpoint
    export RDS_PORT=3306
   

2. Run source ~/.bash_profile

3. Now you can deploy your project.

Solution 3

1. Set all environment properties in EB environment's configuration. (Go to Configuration->Software->Edit->Environment properties and add the key and values).

2. Add this snippet at the beginning of settings.py

    from pathlib import Path
    import os
    import subprocess
    import ast


    def get_environ_vars():
        completed_process = subprocess.run(
            ['/opt/elasticbeanstalk/bin/get-config', 'environment'],
            stdout=subprocess.PIPE,
            text=True,
            check=True
        )

        return ast.literal_eval(completed_process.stdout)

3. Go to Database section and replace it with this snippet

    if 'RDS_HOSTNAME' in os.environ:
        DATABASES = {
            'default': {
            '    ENGINE': 'django.db.backends.mysql',
                 'NAME': os.environ['RDS_DB_NAME'],
                 'USER': os.environ['RDS_USERNAME'],
                 'PASSWORD': os.environ['RDS_PASSWORD'],
                 'HOST': os.environ['RDS_HOSTNAME'],
                 'PORT': os.environ['RDS_PORT'],
        }
    }
    else:
        env_vars = get_environ_vars()
        DATABASES = {
            'default': {
            'ENGINE': 'django.db.backends.mysql',
            'NAME': env_vars['RDS_DB_NAME'],
            'USER': env_vars['RDS_USERNAME'],
            'PASSWORD': env_vars['RDS_PASSWORD'],
            'HOST': env_vars['RDS_HOSTNAME'],
            'PORT': env_vars['RDS_PORT'],
        }
    }
   

4. Deploy the project.

5. Login to the environment (eb ssh environment_name).

6. Go to /var/app/current/ directory and run this command: source /var/app/venv/*/bin/activate.

7. Finally run python3 manage.py migrate.

Conclusion:

Solution 1 is little complex and secret manager is not free (30 days trial only).
Solution 2 is simplest one but I do not recommend tempering any file manually on EB.
Solution 3 is a clean solution which I will use. This solution also takes care of this bug fix in future.