predict() got an unexpected keyword argument 'stats'

Question

I am trying to get predictions from a Tensor Flow custom routine served on AI platform.

I have managed to serve it with the following settings: --runtime-version 2.3 --python-version 3.7 --machine-type mls1-c4-m2

But I keep getting this error when i try to make any predictions.

ERROR:root:Prediction failed: predict() got an unexpected keyword argument 'stats'
ERROR:root:Prediction failed: unknown error.

The routine has two steps:

1. Takes the input (a string) and transforms it into a embedding using a bow model in .pkl format
2. Uses the embeddings for getting the predictions using a keras model saved as an .h5 file

this is my setup.py

from setuptools import setup

REQUIRED_PACKAGES = ['Keras==2.3.1', 'sklearn==0.0', 'h5py<3.0.0', 'numpy==1.16.0', 'scipy==1.4.1', 'pyyaml==5.2']

setup(
        name='my_custom_code',
        version='0.1',
        scripts=['predictor.py'],
        install_requires=REQUIRED_PACKAGES,
        packages=find_packages(),
        include_package_data=False,
        description=''
) 

And this is my predictor.py

import os
import pickle
import tensorflow as tf
import numpy as np

class MyPredictor(object):

    def __init__(self, model, bow_model):
        self._model = model
        self._bow_model = bow_model

    def predict(self, instances):

        outputs = []

        for x in instances:
            vector = self.embedding(x)
            output = self._model.predict(vector)
            outputs.append(output)

        return outputs

    def embedding(self, statement):
        vector = self._bow_model.transform(statement).toarray()
        vector = vector.to_list()
        return vector


    @classmethod
    def from_path(cls, model_dir):

        model_path = os.path.join(model_dir, 'model.h5')
        model = tf.keras.models.load_model(model_path, compile = False)

        preprocessor_path = os.path.join(model_dir, 'bow.pkl')
        with open(preprocessor_path, 'rb') as f:
            bow_model = pickle.load(f)

        return cls(model, bow_model)

The script im using for testing is:

import googleapiclient.discovery

instances = ['test','test']]

service = googleapiclient.discovery.build('ml', 'v1')
name = 'projects/{}/models/{}/versions/{}'.format(PROJECT_ID, MODEL_NAME, VERSION_NAME)

response = service.projects().predict(
    name=name,
    body={'instances': instances}
).execute()

if 'error' in response:
    raise RuntimeError(response['error'])
else:
  print(response['predictions'])

Answer 1

According to the Custom prediction routine documentation, once creating the predictor class, predict() method should be supplied with self, instances, **kwargs arguments to properly handle the prediction request.

instances: A list of prediction input instances.

**kwargs: A dictionary of keyword args provided as additional fields on the predict request body.