Search code examples
pythonserverjupyter-notebookbackend

Is jupyter reliable enough to run a small web server?

I'm a firmware engineer and a newbie to web development.

I was recently tasked with adding functionality to our internal business systems at the request of our sales team. Any modification to an existing system must be approved by the system administrator each time. Since there is no separate test system, it is risky to develop directly on the system.

Fortunately, the system supports external communication via REST APIs. I decided to configure a web server to implement the functionality.

The features we want to implement are:

  1. Provide UI via a simple webpage

  2. Receive, process, and send data via a REST API

For security reasons, I need to use only the internal network. Therefore, I have the following options to build a web server:

  1. Get a virtual machine instance
  2. Get a Jupyter lab page, similar to a Google Colab page.

I know that getting a VM Instance gives me more freedom to do more things. However, VM Instances have a very cumbersome application process, whereas Jupyter is much simpler.

From here, it's the body.

  • I want to run a web server via ipynb script.

  • The webpage will have fewer than 100 concurrent users. The amount of data to process is also small, ranging from a few KB to a few MB per query.

  • I think I'll use django, but I'm not sure. I could just use the built-in modules only.

  • Sales teams, of course, don't have programmers. If the server dies often, the team leader will be angry, and I'll be sad.

If I run a web server for a long time through an ipynb script on a Jupyter lab, what problems can I expect? I am only interested in stability issues. Other issues such as port opening can be excluded.

e.g.)

  • Jupyter might leak RAM and eventually shut down.
  • Performance might be very low, and the script might terminate when the number of visitors to the web page exceeds 10.
  • ipynb is not intended for long-term use; the server will automatically shut down after a few hours.

And can you tell me a solution?

e.g.) There are auto-restart option for jupyter somewhere.

Thanks for considering my question.

Solution

  • Yes, you can run a small web server from the Jupyter Notebook. Is it a good practice? No.

    Do you want to start a web server just to make REST endpoint for your code in the notebook?

    For running notebook as REST endpoint please check Jupyter Kernel Gateway https://jupyter-kernel-gateway.readthedocs.io/en/latest/ - you can create REST endpoints in the notebook.

    For sharing notebooks you can check Mercury framework https://github.com/mljar/mercury - you can share notebooks as interactive web apps. There is a plan to support notebooks as REST API as well (GitHub issue for tracking progress).