gunicorn with compiled python source

Question

I have compiled my python files including gunicorn with PyInstaller on centOS 7. I'm trying to run those executables on another centOS 7 machine I'm getting ModuleNotFoundError: No module named 'manage' but my module manage is present in the same directory in executable form. Here is the full command and its output.

./gunicorn -c /opt/myproject/configuration/gunicorn_conf.ini --bind unix:/etc/myproject/myproject.sock -m 007 manage:app --preload

!!!
!!! WARNING: configuration file should have a valid Python extension.
!!!

Traceback (most recent call last):
  File "gunicorn", line 8, in <module>
  File "site-packages/gunicorn/app/wsgiapp.py", line 58, in run
  File "site-packages/gunicorn/app/base.py", line 228, in run
  File "site-packages/gunicorn/app/base.py", line 72, in run
  File "site-packages/gunicorn/arbiter.py", line 58, in __init__
  File "site-packages/gunicorn/arbiter.py", line 118, in setup
  File "site-packages/gunicorn/app/base.py", line 67, in wsgi
  File "site-packages/gunicorn/app/wsgiapp.py", line 49, in load
  File "site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
  File "site-packages/gunicorn/util.py", line 358, in import_app
  File "importlib/__init__.py", line 127, in import_module
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'manage'
[20316] Failed to execute script gunicorn

Am I missing something here? or gunicorn does not work with compiled files? if it does not, then is there any alternative for it other than using the source?

Answer 1

I had an issue similar where the import failed for gunicorn.glogger. I solved it by adding the import to the top of the module explicitly.

from gunicorn import glogger

I am using gunicorn 20.0.4 and I do not see a module called manage. Is it possible this is a dependency from another library?

Another possible Fix

Hidden imports are specified in the pyinstaller docs, you can use the switch --hidden-import in your build command to include implicitly used libraries within the binary. If you have multiple imports you can use a : to separate the list.

Invoking GUnicorn

Green unicorn is built to be started from the command line. This is easy to do within a docker container or by running the command on a host machine. Unfortunately this is not the case when using pyinstaller as you need a "python" entry point. I had to build a custom application from the gunicorn docs (copy/paste) to allow my application to start up as a script. From here I was able to tell pyinstaller to use this script to start the server under if __name__ == '__main__':

See example below.

Bugs

Even with the binary built with required dependencies I ran into an issue where the web application would stop responding after a random number of requests. I did not investigate further as it pushed me into another direction of building an RPM for easy installation.

Update February 4th, 2021

Setting arguments to GUnicorn.

import gunicorn.app.base


class StandaloneApplication(gunicorn.app.base.BaseApplication):

    def __init__(self, app, options=None):
        self.options = options or {}
        self.application = app
        super().__init__()

    def load_config(self):
        config = {key: value for key, value in self.options.items()
                  if key in self.cfg.settings and value is not None}
        for key, value in config.items():
            self.cfg.set(key.lower(), value)

    def load(self):
        return self.application


if __name__ == '__main__':
    options = {
        'bind': 'unix:/etc/myproject/myproject.sock',
        'preload': True,
        'config': '/opt/myproject/configuration/gunicorn_conf.ini',
        'umask': '007',
        '<argument name>': '<argument value>',
    }
    StandaloneApplication(<your flask application object or factory>, options).run()

Haven't tested this specific setup but I am using something very similar. Each of the GUnicorn switch/option in the CLI maps directly to python variables, you can find that here. These options are set in our options dictionary which the server will accept as the second augument in the StandaloneApplication constructor. You will need to pass in the app object as the first argument or application factory function from your manage module.

I do recommend looking at python configuration files. You can also export these as environment variables and use os.getenv("<env name>") to access them. It just keeps the server from using hardcoded values that other users will have to change in the source.