I want to classify documents in 4 categories (locations) based on 3 columns, burks a 4 digit code, lifnr a dictionary value, and waers, also a dictionary value with a LinarClassifier. Then save the model, serve it, and throw burks, lifnr and waers values at it to get a prediction.
My training data looks like this:
bukrs;lifnr;waers;location
5280;1004008999;EUR;0
5280;1004009000;EUR;2
5280;1004003061;EUR;1
...
And I can successfully train the model, and save it, which results in a saved_model.pb and Variables folder.
So far so good.
I have checked if the model itself is working like this:
saved_model_cli show --dir 1561324458 --all
which gives me:
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['classification']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['classes'] tensor_info:
dtype: DT_STRING
shape: (-1, 4)
name: head/Tile:0
outputs['scores'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 4)
name: head/predictions/probabilities:0
Method name is: tensorflow/serving/classify
signature_def['predict']:
The given SavedModel SignatureDef contains the following input(s):
inputs['examples'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['all_class_ids'] tensor_info:
dtype: DT_INT32
shape: (-1, 4)
name: head/predictions/Tile:0
outputs['all_classes'] tensor_info:
dtype: DT_STRING
shape: (-1, 4)
name: head/predictions/Tile_1:0
outputs['class_ids'] tensor_info:
dtype: DT_INT64
shape: (-1, 1)
name: head/predictions/ExpandDims:0
outputs['classes'] tensor_info:
dtype: DT_STRING
shape: (-1, 1)
name: head/predictions/str_classes:0
outputs['logits'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 4)
name: linear/linear_model/linear/linear_model/linear/linear_model/weighted_sum:0
outputs['probabilities'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 4)
name: head/predictions/probabilities:0
Method name is: tensorflow/serving/predict
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['classes'] tensor_info:
dtype: DT_STRING
shape: (-1, 4)
name: head/Tile:0
outputs['scores'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 4)
name: head/predictions/probabilities:0
Method name is: tensorflow/serving/classify
and that looks good to me.
Here is my whole training python script:
from __future__ import absolute_import, division, print_function, unicode_literals
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
import os
import pandas as pd
import re
import seaborn as sns
from tensorflow import feature_column
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
import itertools
from itertools import islice
#read data
dataframe = pd.read_csv('invoices_classed2.csv', sep=';',header=0)
dataframe.head()
#cut in sets
train, test = train_test_split(dataframe, test_size=0.3)
train, val = train_test_split(train, test_size=0.3)
#print metrics
print(len(train), 'train examples')
print(len(val), 'validation examples')
print(len(test), 'test examples')
# A utility method to create a tf.data dataset from a Pandas Dataframe
labels = pd.Series();
def df_to_dataset(dataframe, shuffle=False, batch_size=32):
dataframe = dataframe.copy()
labels = dataframe.pop('location')
ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels))
if shuffle:
ds = ds.shuffle(buffer_size=len(dataframe))
ds = ds.batch(batch_size)
return ds
# A utility method to create a tf.data dataset from a Pandas Dataframe and use it as functional variable
def make_input_fn(dataframe=None, n_epochs=None, shuffle=False, batch_size=32):
def input_fn():
internal_dataframe = dataframe.copy()
labels = internal_dataframe.pop('location')
ds = tf.data.Dataset.from_tensor_slices((dict(internal_dataframe), labels))
if shuffle:
ds = ds.shuffle(buffer_size=len(internal_dataframe))
ds = ds.repeat(n_epochs)
ds = ds.batch(batch_size)
return ds
return input_fn
#building feature columns
bukrs = feature_column.numeric_column("bukrs")
lifnr = feature_column.categorical_column_with_vocabulary_list(
'lifnr',['1004000409','1004003061','1004008999','1004009001','1004009000','1004003768','1004009002'])
lifnr_one_hot = feature_column.indicator_column(lifnr)
waers = feature_column.categorical_column_with_vocabulary_list(
'waers', ['EUR', 'GBP', 'USD','JPY','CZK','HUF'])
waers_one_hot = feature_column.indicator_column(waers)
actual_feature_columns = []
actual_feature_columns.append(bukrs)
actual_feature_columns.append(lifnr_one_hot)
actual_feature_columns.append(waers_one_hot)
#making datasets
train_ds = make_input_fn(train)
val_ds = make_input_fn(val)
test_ds = make_input_fn(test)
print ('####################creating model####################')
linear_est = tf.estimator.LinearClassifier(feature_columns=actual_feature_columns,n_classes=4,model_dir="C:\\Users\\70D4867\\Desktop\\invoicemodel")
print ('####################Train model####################')
#Train model.
linear_est.train(train_ds,max_steps=10000)
print ('####################Evaluation####################')
# Evaluation.
result = linear_est.evaluate(val_ds, steps=1000)
print ('####################printing result####################')
print(result)
print ('####################Done evaluating####################')
for key in sorted(result):
print (key, result[key])
print ('####################predictions####################')
y_generator = linear_est.predict(test_ds)
print ('####################slice predictions####################')
predictions = list(itertools.islice(y_generator,len(test)))
print ('####################predictions output####################')
final_preds = []
template = ('\nPrediction is "{}" ({:.1f}%)')
i = 0;
for pred in (predictions):
final_preds.append(pred['class_ids'][0])
class_id = pred['class_ids'][0]
probability = pred['probabilities'][class_id]
i = i +1
expected = []
for index, row in test.iterrows():
expected.append(row['location'])
print ('####################Test Results####################')
print(classification_report(expected,final_preds))
print ('####################Saving Model####################')
feature_spec = tf.feature_column.make_parse_example_spec(actual_feature_columns)
print(feature_spec)
my_serving_input_receiver_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)
linear_est.export_saved_model(export_dir_base="invoicemodel\\1",serving_input_receiver_fn=my_serving_input_receiver_fn)
But when I want to get a prediction from the model like this:
saved_model_cli run --dir invoicemodel\1\1561324458 --tag_set serve --signature_def predict --input_examples 'examples=[{"bukrs": 5280, "lifnr": "1004003930", "waers": "EUR"}]'
I would expect something like:
[0]
I get the error:
NameError: name 'bukrs' is not defined
I also tried to feed it a .npy file. I created the file out of some potion of my training data:
bukrs;lifnr;waers
5280;1004008999;EUR
5280;1004009000;EUR
5280;1004003061;EUR
...
like this:
csv_fn = "invoices_classed_npy.csv"
file = pd.read_csv(csv_fn)
np.save('invoices_classed_npy.npy', file, allow_pickle = True);
But when I tried:
saved_model_cli run --dir .\invoicemodel\1\1561324458 --tag_set serve --signature_def classification --inputs 'inputs="invoices_classed_npy.npy"'
I expected
[1],[2],[0]
I got:
ValueError: Cannot feed value of shape (55276, 1) for Tensor
'input_example_tensor:0', which has shape '(None,)'
Okay so I also tried to serve it in a container:
docker run -t --rm -p 8501:8501 \
> -v "/data/container/tensorflow/model:/models/saved_model" \
> -e MODEL_NAME=saved_model \
> tensorflow/serving
and I got:
/usr/bin/tf_serving_entrypoint.sh: line 3: 6 Illegal instruction (core dumped) tensorflow_model_server --port=8500 --rest_api_port=8501 --model_name=${MODEL_NAME} --model_base_path=${MODEL_BASE_PATH}/${MODEL_NAME} "$@"
What am I doing wrong? How do I correctly get predictions from my model?
I figured out what my problem was.
You have to specify the serving_input_receiver_fn before you train the model, and somehow pulling it from the feature column does not work.
this was the part that did not work at the end of my script:
feature_spec = tf.feature_column.make_parse_example_spec(actual_feature_columns)
my_serving_input_receiver_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)
linear_est.export_saved_model(export_dir_base="invoicemodel\\1",serving_input_receiver_fn=my_serving_input_receiver_fn)
This led to a signature_def of predict that looked like:
signature_def['predict']:
The given SavedModel SignatureDef contains the following input(s):
inputs['examples'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
and to the Error:
NameError: name 'bukrs' is not defined
I changed the definition of the serving_input_receiver_fn like this
feature_placeholder = {'bukrs': tf.placeholder(tf.int32, [1],
name='bukrs_placeholder'),'lifnr': tf.placeholder('string', [1],
name='lifnr_placeholder'),'waers': tf.placeholder('string', [1],
name='waers_placeholder')}
my_serving_input_receiver_fn =
tf.estimator.export.build_raw_serving_input_receiver_fn(feature_placeholder)
####now train the model####
linear_est.train(train_ds,max_steps=10000)
#evaluation ...
#test ....
#### then save at the end ###
linear_est.export_saved_model(export_dir_base="invoicemodel\\1",serving_input_receiver_fn=my_serving_input_receiver_fn)
that led to the correct signature_def of the model:
signature_def['predict']:
The given SavedModel SignatureDef contains the following input(s):
inputs['bukrs'] tensor_info:
dtype: DT_INT32
shape: (-1)
name: bukrs_placeholder:0
inputs['lifnr'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: lifnr_placeholder:0
inputs['waers'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: waers_placeholder:0
and now I can get predictions from the model:
saved_model_cli run --dir \1\1561727347 --tag_set serve --signature_def predict --input_exprs="bukrs=[5280];lifnr=['1004002578'];waers=['EUR']"
Which results in a prediction:
Result for output key all_class_ids:
[[0 1 2 3]]
Result for output key all_classes:
[[b'0' b'1' b'2' b'3']]
Result for output key class_ids:
[[0]]
Result for output key classes:
[[b'0']]
Result for output key logits:
[[493.99664 475.6605 482.86667 483.922 ]]
Result for output key probabilities:
[[9.99943256e-01 1.08814335e-08 1.46652310e-05 4.21320110e-05]]
I hope this helps someone.