Convert Image to TensorFloat in Windows ML (For converted Keras model)

Question

I followed this official tutorial to create an UWP app which can detect handwritten digits using pre-trained model.

But I need to use Keras models.

So I converted this Keras model from the Keras examples which is also trained on the MNIST dataset with winmltools in python to ONNX:

onnx_model = winmltools.convert_keras(model, 7, name='mnist')   
print(onnx_model.graph.input)   
winmltools.save_model(onnx_model, "mnist_tf.onnx")`

When I Import this in Visual Studio mlgen generates this code for the Input:

public TensorFloat conv2d_1_input; // shape(-1,28,28,1)

But on the original model from the tutorial it is:

public ImageFeatureValue Input3; // BitmapPixelFormat: Gray8, BitmapAlphaMode: Premultiplied, width: 28, height: 28

How can I convert an ImageFeatureValue to TensorFloat?

I also need to import image files but I only found ways which also used ImageFeatureValue and not TensorFloat.

Thank you!

Answer 1

you have a few options.

1. You can modify the generated code to accept an ImageFeatureValue as the mnistInput. Note that ImageFeatureValue can be accepted by the Bind call an converted to a TensorFloat as long as the shapes are compatible.
2. You can use the WinMLDashboard tool to add Image Denotation and image metadata to the model. I don't see the Keras model you linked to, but I'm assuming it doesn't have any of the image metadata that the mnist model from the tutorial has. Set the "Type denotation" to "Image" like this:

and the metadata to this:

where the NominalRange is 0-255. That is how the model in the tutorial is set up.