Manage a python application using pm2

Question

I have a Python 3 application with the following structure:

application
  |
  - application.json
  |
  - package
  |  |
  |  - __init__.py
  |  - module1.py
  |  - module2.py
  |
  - test_package
  |  |
  |  - __init__.py
  |  - test_module1.py
  |  - test_module2.py
  |
  - conf
  |  |
  |  - package.conf
  |
  - log
  | |
  | - module1.log
  | - module2.log
  |
  - virtualenv    (Python 3 virtual environment)
    |
    - bin
    | |
    | - python
    | - activate
    | - ...
    |
    - ...

It needs to be launched as follows from the application directory (in the example assume your working directory is /path/to/application/):

/path/to/application/virtualenv/bin/python -m package.module1

or

virtualenv/bin/python -m package.module1

This application runs a back-end service on a server, and I would like to manage it with PM2, since I already use it to manage a Node.js based web site in the server.

I have tried using the following environment file (application.json) without success, and I can't figure out what is wrong with it:

{
    "name": "application",
    "script": "/path/to/application/virtualenv/bin/python",
    "args": "-u -m package.module1",
    "cwd": "/path/to/application/",
    "instances": 1,
    "autorestart": true,
    "watch": true,
    "max_memory_restart": "1G",
    "pid": "/path/to/application/log/application.pid",
    "out_file": "/path/to/application/log/application_out_file.log",
    "error_file": "/path/to/application/log/application_error_file.log"
}

When I run the pm2 start application.json, the app starts, crashes after one iteration of its inner loop, and then it starts restarting and crashing over and over again before completing a single iteration of the inner loop. Then, I run an pm2 stop application command, and the application starts running as it should ¯_(ツ)_/¯. Running the pm2 stop application command again does stop the app, and delete works too.

I also tried to use the interpreter and interpreter-args as stated in the PM2 documentation, but I am already calling the interpreter in my script, so that does not work.

How should I call my script so that I don't get this weird behavior? What is the recommended way to call a python script that needs to be called using python -m package.module?

Edit

Using the following command from the application directory seems to work:

pm2 start '/path/to/application/virtualenv/bin/python -u -m package.module1'

How can I translate this into a configuration file for PM2?

Answer 1

I found out what was the issue. The environment file (application.json) had the watch field set to true. This setting watches if there are any file changes in the application's folder or subfolders (for my example, the application folder and subfolders). Here is the documentation on that feature.

The logs are included in the folder application/log/, which is a subfolder of application. As soon as anything was written to the logs, PM2 detected that, and it reloaded the application. Disabling this setting fixed all the issues.