Cannot load a saved '.h5' checkpoint file into a python app, when deploying into Heroku

Question

I have been trying to create a machine learning app using Python. In that case I need to import a pre-trained model stored as checkpoint-1.111.h5 file in the repository. But I am unable to import that model when deploying the code into Heroku. The repository gets deployed successfully, but on opening the app it shows

Internal Server Error

The code for the app.py is as follows:

import re
from flask import Flask, jsonify, render_template, request

from keras.models import load_model
# Run this cell to mount your Google Drive.
#from google.colab import drive
from keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences




app = Flask(__name__)

@app.route('/')
def index():
    best_model =  load_model('checkpoint-1.111.h5')
    data_int_t = pad_sequences([[1, 72, 19, 38], [], [], [], []],padding='pre', maxlen=(30-5))
    data_test = pad_sequences(data_int_t, padding='post', maxlen=(30))
    y_prob = best_model.predict(data_test)
    return str(x + y)


if __name__ == '__main__':
    app.run()

On opening the app I always obtain the result Internal server error. The logs are as follows:

2019-07-02T17:38:32.982084+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.6/site-packages/keras/utils/io_utils.py", line 186, in __init__
2019-07-02T17:38:32.982086+00:00 app[web.1]:     self.data = h5py.File(path, mode=mode)
2019-07-02T17:38:32.982088+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.6/site-packages/h5py/_hl/files.py", line 394, in __init__
2019-07-02T17:38:32.982090+00:00 app[web.1]:     swmr=swmr)
2019-07-02T17:38:32.982092+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.6/site-packages/h5py/_hl/files.py", line 170, in make_fid
2019-07-02T17:38:32.982094+00:00 app[web.1]:     fid = h5f.open(name, flags, fapl=fapl)
2019-07-02T17:38:32.982097+00:00 app[web.1]:   File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
2019-07-02T17:38:32.982099+00:00 app[web.1]:   File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
2019-07-02T17:38:32.982101+00:00 app[web.1]:   File "h5py/h5f.pyx", line 85, in h5py.h5f.open
2019-07-02T17:38:32.982110+00:00 app[web.1]: OSError: Unable to open file (file signature not found)

Both the files app.py and checkpoint-1.111.h5 are present in the same location of the repository. However the checkpoint-1.111.h5 is actually a git-lfs file. But is that an issue? keras.models.load_model totally works in loading the checkpoint-1.111.h5 file when I use the same code in Google Colab.

Answer 1

the checkpoint-1.111.h5 is actually a git-lfs file

Heroku doesn't support Git LFS and unfortunately I don't know enough about TensorFlow or Keras to suggest an alternative solution. With its hard limit on slug size and ephermal filesystem Heroku might not be a good fit for this application.