Search code examples
c#unity-game-engineemgucvunity3d-gui

How to convert EmguCV Image to Unity3D Sprite?


I try to render image from the Emgu's camera capture on new Unity3D UI system. Till now, I used ImageToTexture2d from this repository: https://github.com/neutmute/emgucv/blob/3ceb85cba71cf957d5e31ae0a70da4bbf746d0e8/Emgu.CV/PInvoke/Unity/TextureConvert.cs and then used Sprite.Create() to finally achieve the wanted result.

BUT! It appears there is some massive memory leak as after 2-3 minutes of my game running Unity editor suddenly takes about 3GB of RAM where it started with about 200MB.

I have two susspects:

  1. (More Probabble) The method I'm using does not clean the memory. It uses InterOp and creates some unsafe pointers - it smells leakage.
  2. The Sprite.Create runned every frame holds old sprites in memory and does not remove them.

Does any of You knows any other way to convert Emgu's Image to Sprite/Texture(without using an InterOp) or any other way I could show it on New Unity's UI. It has to be the Emgu's Image as I also do some operations on the images I recieve from camera.

Thanks in advance for responses and help. :D


Solution

  • After some research I've found what was the problem, but didn't have time to describe it. I didn't knew that Textures created each frame are held somewhere in engine. They must be destroyed before generating a new one from Emgu Image.

    Here's a part of my code used in my project:

    //Capture used for taking frames from webcam
    private Capture capture;
    //Frame image which was obtained and analysed by EmguCV
    private Image<Bgr,byte> frame;
    //Unity's Texture object which can be shown on UI
    private Texture2D cameraTex;
    
    //...
    
    if(frame!=null)
        frame.Dispose();
    frame = capture.QueryFrame();
    if (frame != null)
    {
        GameObject.Destroy(cameraTex);
        cameraTex = TextureConvert.ImageToTexture2D<Bgr, byte>(frame, true);
        Sprite.DestroyImmediate(CameraImageUI.GetComponent<UnityEngine.UI.Image>().sprite);
        CameraImageUI.sprite = Sprite.Create(cameraTex, new Rect(0, 0, cameraTex.width, cameraTex.height), new Vector2(0.5f, 0.5f));
    }