How to deploy a Node.js application with a child process (python) in Heroku?

I'm trying to deploy a Node.js application with a child process that runs a machine learning algorithm. I can use this locally, but when I try to run at the Heroku server I recive some messages calling that are some libraries missing, like bellow:

ModuleNotFoundError: No module named 'pandas'

I tried to create manually the requirements.txt and put the necessary libraries there:

pandas
pymongo
dnspython
scikit-learn      
scipy           
selenium
webdriver-manager
textblob

But it doesn't work. Do I need to do some extra configuration?

Thank you so much for your help!

Solution

The way your Heroku dynos run your software is through something called a buildpack.

When you deploy an application to Heroku, it looks at your code and tries to figure out which programming language you are using, then based on that, will run your app using the corresponding buildpack.

For example, if you deploy an app to Heroku and the app has a package.json file in the root of your project directory, Heroku will assume your app is a JavaScript app and use the Node.js buildpack.

Buildpacks contain a number of pre-installed dependencies. For example, the Node.js buildpack contains node (so you can run your JavaScript code) as well as a number of Linux dependencies so that your app will be able to install common libraries/tools that might rely on them.

But... One downside of this buildpack strategy is that if you're deploying a Node.js app, for example, the default Node.js building will NOT come with Python and the various Python library dependencies installed. This is because Heroku supports a lot of different programming environments, and it would be slow/complex if there was just a single buildpack that had EVERYTHING installed. It'd be crazy!

So what you need to do, in your case, is use multiple buildpacks! Heroku has a way for you to enable multiple buildpacks for your app so that your app can have the Node.js dependencies as well as the Python dependencies, for example!

This article on Heroku's documentation site explains how to use multiple buildpacks for a given app.

Here are the specific instructions for simplicity's sake:

# This command will set your default buildpack to Node.js
$ heroku buildpacks:set heroku/nodejs

# This command will set it up so that the Heroku Python buildpack will run first
$ heroku buildpacks:add --index 1 heroku/python

By doing the above, you'll be able to have Heroku install your Python dependencies via a traditional requirements.txt file like you would with any normal Python application.