Passing Images to opencv from unity

Question

I've created a dll to pass two images from unity to opencv and return a struct. But I'm getting a run time c++ error. This is the code I used in C++:

    struct rigidTransform
    {
        rigidTransform(double eular_angle_x, double eular_angle_y, double eular_angle_z, double eular_dis_x, double eular_dis_y, double eular_dis_z) : Eular_angle_x(eular_angle_x), Eular_angle_y(eular_angle_y), Eular_angle_z(eular_angle_z), Eular_dis_x(eular_dis_x), Eular_dis_y(eular_dis_y), Eular_dis_z(eular_dis_z) {}
        double Eular_angle_x;
        double Eular_angle_y;
        double Eular_angle_z;

        double Eular_dis_x;
        double Eular_dis_y;
        double Eular_dis_z;
    };
        struct Color32
    {
        uchar r;
        uchar g;
        uchar b;
        uchar a;
    };

    extern "C" rigidTransform __declspec(dllexport) __stdcall findPose(Color32* img1, Color32* img2,  int width, int height)
    {
    Mat Img1(height, width, CV_8UC4, img1), Img2(height, width, CV_8UC4, img2);
    //my pose estimation code here

    return Tr;
    }

And my C# code used in unity is:

    using System.Collections;
    using System.Collections.Generic;
    using System.Runtime.InteropServices;
    using UnityEngine;

    public struct regidTransform
    {
        public double eular_angle_x;
        public double eular_angle_y;
        public double eular_angle_z;
        public double eular_dis_x;
        public double eular_dis_y;
        public double eular_dis_z;
    }



public class UI : MonoBehaviour
{
    [DllImport("test2", EntryPoint = "findPose", CallingConvention = CallingConvention.Cdecl)]
    public unsafe static extern regidTransform findPose(Color32[] img1, Color32[] img2, int width, int height);

    public regidTransform results_;


    public WebCamTexture webcamTexture;
    public Color32[] img1;
    public Color32[] img2;

    void Start()
    {

        results_ = new regidTransform();




        webcamTexture = new WebCamTexture();
        webPlane.GetComponent<MeshRenderer>().material.mainTexture = webcamTexture;
        webcamTexture.Play();

        if(webcamTexture.isPlaying)
        {
            img1 = webcamTexture.GetPixels32();
            System.Array.Reverse(img1);
            img2 = webcamTexture.GetPixels32();
            System.Array.Reverse(img2);


            unsafe
            {

                    results_ = findPose(img1, img2, webcamTexture.width, webcamTexture.height);


            }
        }




    }


    void Update()
    {


    }
}

When I run this code, it throws an error saying:

Runtime Error!

Program:

This Application has requested the Runtime to terminate it in an unusual way. Please contact the application support team.

Kindly assist me on what wrong I've made in this code.

Thanking You in advance!

UPDATED:

With the help of @Programmer I found that the code is getting crashed when calling calcOpticalFlowPyrLK(). And when checked if I'm getting the image correctly using imshow() function, I found it results a complete black image.

Answer 1

Finally I figured it out.

Yes that code worked well. and there is no error in my C++ code or C# code. The reason that resulted a complete black image for me is, I call my dll fuction in side strat() function of unity, which only execute once. So I found in my case, that the typical 1st image acquired to dll will mostly a black one, this was proved when I tested with imshow() function inside the dll. The reason it throws error is because I'm passing that black image to calcOpticalFlowPyrLK() function, which doesn't have any feature points to track.

So I created another function inside dll to check if the true image is acquired before passing it to my code by checking for feature points.

Something like:

featureDetection(frame_1, points1);

if (points1.size() > MIN_NUM_FEATURES)
{
    //Pass the frame to my code.
}

Now things will perfectly work. My sincere thanks to the @Programmer who helped me to figure this out.