Handling ONNX/ML.NET models with generic interfaces

i'm kind of struggling with a ML.NET related problem here and am hoping that someone might can help me.

I'm developing an (.NET core) application which consumes ONNX models whose inputs are unknown at compile time. What I've done so far:

I'm able to compile an assembly during runtime which contains the input class definition and load this definition:

        var genericSampleAssembly =
            AssemblyLoadContext.Default.LoadFromAssemblyPath("/app/storage/sample.dll");
        Type genericInputClass = genericSampleAssembly.GetType("GenericInterface.sample");

Also I'm able to train a model with that dynamically created Inputtype using Reflection:

        MethodInfo genericCreateTextLoader = typeof(TextLoaderSaverCatalog).GetMethods()
            .Where(_ => _.Name == "CreateTextLoader")
            .Single(_ => _.GetParameters().Length == 6)
            .MakeGenericMethod(_genericInputClass);

        TextLoader reader = genericCreateTextLoader.Invoke(_mlContext.Data, new object[] { _mlContext.Data, false, ',', true, true, false}) as TextLoader;

        IDataView trainingDataView = reader.Read("sample.txt");
        var debug = trainingDataView.Preview();

        var pipeline = _mlContext.Transforms.Concatenate("Features", _featureNamesModel
            .AppendCacheCheckpoint(_mlContext)
            .Append(_mlContext.Regression.Trainers.StochasticDualCoordinateAscent(labelColumn: "Label",
                featureColumn: "Features")));

        ITransformer model = pipeline.Fit(trainingDataView);

But I'm not able to make predictions by now because I don't know how to invoke the PredictionEngine. I'm able to get a generic version of that CreatePredictionEngine method but don't now how to cast that returning object to an PredictionEngine and finally invoke the Predict method:

        MethodInfo genericCreatePredictionEngineMethod = typeof(PredictionEngineExtensions).GetMethods()
            .Single(_ => _.Name == "CreatePredictionEngine")
            .MakeGenericMethod(new Type[] { genericInputClass, typeof(GenericPrediction)});

        var predictionEngine = genericCreatePredictionEngineMethod.Invoke(_model, new object[] {_model, _mlContext, null, null});

predictionEngine is of Type object in this case but I need to cast it to something like PredictionEngine<genericInputClass, GenericPrediction>, while genericInputClass is the class from that dynamically created assembly and GenericPrediction is a simple class with one output I know at compile time.

So whats missing is something like:

        MethodInfo genericCreatePredictionEngineMethod = typeof(PredictionEngineExtensions).GetMethods()
            .Single(_ => _.Name == "CreatePredictionEngine")
            .MakeGenericMethod(new Type[] { genericInputClass, typeof(GenericPrediction)});

        PredictionEngine<genericInputClass, GenericPrediction> predictionEngine = genericCreatePredictionEngineMethod.Invoke(_model, new object[] {_model, _mlContext, null, null}) as PredictionEngine<genericInputClass, GenericPrediction>;

        float prediction = predictionEngine.Predict(genericInputClass inputValue);

Does anyone had a similar problem or has any other hints?

I might missed some lines, because I copy/pasted and simplified it pretty quick. In case something is missing, I'll provide it later.

EDIT: I constructed a minimal example to show the basic problem. As mentioned in the comments dynamic isn't possible due to the ML.NET methods.

using System;
using System.Linq;
using System.Runtime.Loader;


namespace ReflectionSample
{
    class Program
    {
        static void Main(string[] args)
        {
            // Example with a known Type
            var extendedClass = new DummyExtendedClass();
            SampleGenericClass<String> sampleGenericClass = extendedClass.SampleGenericExtensionMethod<String>();
            sampleGenericClass.SampleMethod("");

            // At compile time unknown Type - In reality the loaded dll is compiled during runtime
            var runtimeCompiledSampleAssembly =
                AssemblyLoadContext.Default.LoadFromAssemblyPath("C:/Program Files (x86)/Reference Assemblies/Microsoft/Framework/.NETCore/v4.5/System.IO.dll");
            var compileTimeUnknownClass = runtimeCompiledSampleAssembly.GetType("System.IO.TextReader");

            var reflectedExtensionMethod = typeof(Extensions).GetMethods()
                .Single(_=>_.Name== "SampleGenericExtensionMethod")
                .MakeGenericMethod(new[] {compileTimeUnknownClass});

            var howToCastThis = reflectedExtensionMethod.Invoke(extendedClass, new object[] {extendedClass});

            // whats missing:
            // howToCastThis is of Type object but should be SampleGenericClass<System.IO.TextReader>
            // I need to be able to call howToCastThis.SampleMethod(new System.IO.TextReader)
            // I thought this might work via reflecting SampleMethod and MakeGenericMethod

            Console.ReadKey();
        }
    }

    public sealed class SampleGenericClass<T>
    {
        public void SampleMethod(T typeInput)
        {
            Console.WriteLine($"Invoking method worked! T is of type {typeof(T)}");
        }
    }

    public static class Extensions
    {
        public static SampleGenericClass<T> SampleGenericExtensionMethod<T>(this DummyExtendedClass extendedClass)
        {
            Console.WriteLine($"Invoking extension method worked! T is of type {typeof(T)}");
            return new SampleGenericClass<T>();
        }
    }

    public class DummyExtendedClass
    {
        public DummyExtendedClass() { }
    }
}

Greetings Sven

Solution

Nice job on the MCVE. I was able to invoke SampleMethod; it turns out there's not much to it and it's probably less complicated than you imagined.

In your example, the object you're getting, howToCastThis, is of a type that's already close-constructed. As long as you start from that instance's type, you don't need to use MakeGenericMethod.

Let's say you have an object instance, compileTimeTypeUnknownInstance, for the parameter you want to pass to SampleMethod. Since System.IO.TextReader is abstract, compileTimeTypeUnknownInstance would have to be of a concrete, TextReader-derived type. With those conditions satisfied, the following works:

var sampleMethod = howToCastThis.GetType().GetMethods()
    .Single(mi => mi.Name == "SampleMethod");

sampleMethod.Invoke(howToCastThis, new object[] { compileTimeTypeUnknownInstance });

SampleMethod reports that T is of type System.Text.TextReader.

Again, howToCastThis is of a close-constructed type, therefore, so is the method you want.

Note: Although not the case here, a method in a close-constructed type can introduce additional type arguments so you still would have to call MakeGenericMethod to close-construct the method in that case.

Now, if I were to try to translate this to your situation, I'm guessing it would look something like this:

var predictMethod = predictionEngine.GetType().GetMethods()
    .Single(mi => mi.Name == "Predict");

float prediction = (float)predictMethod.Invoke(predictionEngine, new object[] { inputValue });

I'm not sure about the syntax in your pseudocode call to Predict. I assumed that inputValue was the only parameter and genericInputClass was only there to indicate that it was a type argument in the close-constructed type. If this was incorrect, you'll need to figure out what actually goes in that object[] argument.