How to use onnx model in mlnet c#, passing inputs and getting outputs

Question

I trained a model using pytorch I exported it to onnx format and tested in python that it works (it does)

I want to know how I can use this in ml.net in c#

The usage in python looks like this

the model in netorn looks like

I found an example that uses using the packages Microsoft.ML, Microsoft.ML.OnnxRuntime and Microsoft.ML.OnnxTransformer

and is able to use an onnx model but that works with images and since I am very new to this I am unable to figure out how to load the model and make a prediction and get the value of the action which is an array of float size 6.

Answer 1

I found this to be useful for this

you'll also have to define your model schema.

here is the class I ended up with for this model

using Microsoft.ML;
using Microsoft.ML.Data;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Numerics;
using System.Text;
using System.Threading.Tasks;

namespace godot_net_server
{
    class OnnxModelScorer
    {
        private readonly string modelLocation;
        private readonly MLContext mlContext;


        public OnnxModelScorer(string modelLocation, MLContext mlContext)
        {
            this.modelLocation = modelLocation;
            this.mlContext = mlContext;
        }

        public class ModelInput
        {
            [VectorType(10)]
            [ColumnName("input.1")]
            public float[] Features { get; set; }
        }

        private ITransformer LoadModel(string modelLocation)
        {
            Console.WriteLine("Read model");
            Console.WriteLine($"Model location: {modelLocation}");
            
            // Create IDataView from empty list to obtain input data schema
            var data = mlContext.Data.LoadFromEnumerable(new List<ModelInput>());

            // Define scoring pipeline
            var pipeline = mlContext.Transforms.ApplyOnnxModel(modelFile: modelLocation, outputColumnNames: new[] { "31","34" }, inputColumnNames: new[] { "input.1" });

            // Fit scoring pipeline
            var model = pipeline.Fit(data);

            return model;
        }
        public class Prediction
        {
            [VectorType(6)]
            [ColumnName("31")]
            public float[] action{ get; set; }
            [VectorType(1)]
            [ColumnName("34")]
            public float[]  state { get; set; }
        }
        private IEnumerable<float> PredictDataUsingModel(IDataView testData, ITransformer model)
        {
            Console.WriteLine("");
            Console.WriteLine("=====Identify the objects in the images=====");
            Console.WriteLine("");

            IDataView scoredData = model.Transform(testData);

            IEnumerable<float[]> probabilities = scoredData.GetColumn<float[]>("31");
            var a = probabilities.ToList();
            a.Count.ToString();
            return a[0];
        }

        public IEnumerable<float> Score(IDataView data)
        {
            var model = LoadModel(modelLocation);

            return PredictDataUsingModel(data, model);
        }
    }
}

this is how I used it to get a prediction

            MLContext mlContext = new MLContext();

            // Load trained model

            var modelScorer = new OnnxModelScorer("my_ppo_1_model.onnx", mlContext);
            List<ModelInput> input = new List<ModelInput>();
            input.Add(new ModelInput()
            {
                Features = new[]
                {
                    3.036393f,11.0f,2.958097f,0.0f,0.0f,0.0f,0.015607f,0.684984f,0.0f,0.0f
                }
            });
            var action = modelScorer.Score(mlContext.Data.LoadFromEnumerable(input));

The result is as I was expecting it to be