Error Running TensorFlow Serving from Dockerfile

Question

I am working on a container to run TensorFlow Serving.

Here is my Dockerfile:

FROM tensorflow/serving:latest

WORKDIR /

COPY models.config /models/models.config
COPY models/mnist /models/mnist

Here is my models.config for a simple mnist model:

model_config_list {
    config: {
        name: "mnist",
        base_path: "/models/mnist"
        model_platform: "tensorflow"
        model_version_policy {
            specific {
                versions: 1646266834
            }
        }
        version_labels {
            key: 'stable'
            value: 1646266834
        }
    }
}

The models directory is setup as follows:

$ ls -Rls models
total 0
0 drwxr-xr-x  3 david  staff  96 Mar  2 16:21 mnist

models/mnist:
total 0
0 drwxr-xr-x  6 david  staff  192 Mar  2 16:21 1646266834

models/mnist/1646266834:
total 304
  0 drwxr-xr-x  2 david  staff      64 Mar  2 16:21 assets
 32 -rw-r--r--  1 david  staff   15873 Mar  2 16:20 keras_metadata.pb
272 -rw-r--r--  1 david  staff  138167 Mar  2 16:20 saved_model.pb
  0 drwxr-xr-x  4 david  staff     128 Mar  2 16:21 variables

models/mnist/1646266834/assets:
total 0

models/mnist/1646266834/variables:
total 1424
1416 -rw-r--r--  1 david  staff  722959 Mar  2 16:20 variables.data-00000-of-00001
   8 -rw-r--r--  1 david  staff    2262 Mar  2 16:20 variables.index

The problem is that when I build and run my container, I receive an error.

$ docker build -t example.com/example-tf-serving:1.0 .
$ docker run -it -p 8500:8500 -p 8501:8501 --name example-tf-serving --rm example.com/example-tf-serving:1.0

The error is as follows Not found: /models/model:

2022-03-03 00:48:06.242923: I tensorflow_serving/model_servers/server.cc:89] Building single TensorFlow model file config:  model_name: model model_base_path: /models/model
2022-03-03 00:48:06.243215: I tensorflow_serving/model_servers/server_core.cc:465] Adding/updating models.
2022-03-03 00:48:06.243254: I tensorflow_serving/model_servers/server_core.cc:591]  (Re-)adding model: model
2022-03-03 00:48:06.243899: E tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:365] FileSystemStoragePathSource encountered a filesystem access error: Could not find base path /models/model for servable model with error Not found: /models/model not found

How do I fix my Dockerfile so that the above command will work?

For this solution, the quick and easy way will not work for me, so I cannot accept it as a solution:

docker run --name=the_name -p 9000:9000 -it -v "/path_to_the_model_in_computer:/path_to_model_in_docker" tensorflow/serving:1.15.0 --model_name=MODEL_NAME --port=9000

Answer 1

https://www.tensorflow.org/tfx/serving/docker

Optional environment variable MODEL_NAME (defaults to model)

Optional environment variable MODEL_BASE_PATH (defaults to /models)

You are using default values of these env variables, so Tensorflow is trying to find model in /models/model. You have different model path in the container, so /models/model not found is correct.

I would say simple configuration of MODEL_NAME env variable should solve the problem:

$ docker run -it -p 8500:8500 -p 8501:8501 \
  --name example-tf-serving \
  -e MODEL_NAME=mnist \
  --rm example.com/example-tf-serving:1.0

For multiple models https://www.tensorflow.org/tfx/serving/serving_config#model_server_configuration

The easiest way to serve a model is to provide the --model_name and --model_base_path flags (or setting the MODEL_NAME environment variable if using Docker). However, if you would like to serve multiple models, or configure options like polling frequency for new versions, you may do so by writing a Model Server config file.

You may provide this configuration file using the --model_config_file flag and instruct Tensorflow Serving to periodically poll for updated versions of this configuration file at the specifed path by setting the --model_config_file_poll_wait_seconds flag.

See docker doc: https://www.tensorflow.org/tfx/serving/docker#passing_additional_arguments

You need to set CMD in the Dockerfile (so you don't need to specify it in run time, because requirement is to use only Dockerfile), e.g.:

FROM tensorflow/serving:latest

WORKDIR /

COPY models.config /models/models.config
COPY models/mnist /models/mnist

CMD ["--model_config_file=/models/models.config"]