ML.NET : 'Length of memory (691200) must match product of dimensions (3).'

Question

I am having a problem to build a pipeline in ML.NET. When I try to predict, it throws this error:

System.ArgumentException: 'Length of memory (691200) must match product of dimensions (3).'

The pipeline is following:

            var pipeline = _mlContext.Transforms
                .ResizeImages(
                    outputColumnName: "resized_image",
                    imageWidth: 640,
                    imageHeight: 360,
                    inputColumnName: "image")
                .Append(_mlContext.Transforms
                    .ExtractPixels(
                        outputColumnName: ModelConfigParameters.InputColumns.First(),
                        outputAsFloatArray: false,
                        colorsToExtract: ImagePixelExtractingEstimator.ColorBits.Rgb,
                        orderOfExtraction: ImagePixelExtractingEstimator.ColorsOrder.ARGB,
                        interleavePixelColors: false,
                        inputColumnName: "resized_image"))
                .Append(_mlContext.Transforms
                    .ApplyOnnxModel(
                        modelFile: onnxModelPath,
                        outputColumnNames: ModelConfigParameters.OutputColumns,
                        inputColumnNames: ModelConfigParameters.InputColumns));

The input object is:

    public class InputImage
{
    //ToDo:Load image size from some config file
    [ImageType(width: 640, height: 360)]
    [ColumnName("image")]
    public Bitmap Image { get; set; }

}

The prediction object is:

    public class OutputPredictions
{
    [ColumnName("detection_scores:0")]
    public float[] DetectionScores;

    [ColumnName("detection_boxes:0")]
    public float[] DetectionBoxes;

    [ColumnName("detection_classes:0")]
    public float[] DetectionClasses;

    [ColumnName("num_detections:0")]
    public float[] NumDetections;
}

And I do the prediction using:

        var imgPath = @"some/path/to/image.png";
        var img = (Bitmap)Image.FromFile(imgPath);
        var inputData = new InputImage() { Image = img };

        var emptyData = _mlContext.Data.LoadFromEnumerable(new List<InputImage>());
        var model = pipeline.Fit(emptyData);

        var prediction = _mlContext.Model.CreatePredictionEngine<InputImage, OutputPredictions>(model).Predict(inputData);
 

By running it I get the previously mentioned error. When I inspect pipeline using the var preview = pipeline.Preview(emptyData); I get this scheme:

and the input and output of the ONNX model is following:

I don't see the problem. What could it be?

Answer 1

It is a guess but the input image_tensor:0 tensor dimensions are not completely defined in your ONNX model (uint8[unk, unk, unk, 3] as per last screenshot).

691,200 is the size of your image vector: 1 x 3 x 360 x 640. Because only one dimension is fully defined (3) in your ONNX model, I guess the others are ignored.

Try defining them at the ApplyOnnxModel step of the pipeline, using the shapeDictionary parameter:

   .Append(_mlContext.Transforms
                    .ApplyOnnxModel(
                        shapeDictionary: new Dictionary<string, int[]>() 
                        {
                           { "image_tensor:0", new[] { 1, 3, 340, 640} } 
                        },
                        modelFile: onnxModelPath,
                        outputColumnNames: ModelConfigParameters.OutputColumns,
                        inputColumnNames: ModelConfigParameters.InputColumns));

NB: careful with the new dimension: it seems you're sending a [1, 3, 340, 640] vector whereas the model seems to have the 3d as last dimension: [unk, unk, unk, 3].