Search code examples
unity-game-engineimage-processingshadertexture2dcompute-shader

How to modify a Texture pixels from a compute shader in unity?


I stumbled upon a strange problem in vuforia.When i request a camera image using CameraDevice.GetCameraImage(mypixelformat), the image returned is both flipped sideways and rotated 180 deg. Because of this, to obtain a normal image i have to first rotate the image and then flip it sideways.The approach i am using is simply iterating over pixels of the image and modifying them.This approach is very poor performance wise.Below is the code:

    Texture2D image;

    CameraDevice cameraDevice = Vuforia.CameraDevice.Instance;
    Vuforia.Image vufImage = cameraDevice.GetCameraImage(pixelFormat);
    image = new Texture2D(vufImage.Width, vufImage.Height);
    vufImage.CopyToTexture(image);

    Color32[] colors = image.GetPixels32();
    System.Array.Reverse(colors, 0, colors.Length);  //rotate 180deg
    image.SetPixels32(colors);                       //apply rotation
    image = FlipTexture(image);                      //flip sideways






                 //***** THE FLIP TEXTURE METHOD *******//

private Texture2D FlipTexture(Texture2D original, bool upSideDown = false)
{

    Texture2D flipped = new Texture2D(original.width, original.height);

    int width  = original.width;
    int height = original.height;


    for (int col = 0; col < width; col++)
    {
        for (int row = 0; row < height; row++)
        {
            if (upSideDown)
            {
                flipped.SetPixel(row, (width - 1) - col, original.GetPixel(row, col));
            }
            else
            {
                flipped.SetPixel((width - 1) - col, row, original.GetPixel(col, row));
            }
        }
    }

    flipped.Apply();

    return flipped;
}

To improve the performance i want to somehow schedule these pixel operations on the GPU, i have heard that a compute shader can be used, but i have no idea where to start.Can someone please help me write the same operations in a compute shader so that the GPU can handle them, Thankyou!.


Solution

  • The whole compute shader are new for me too, but i took the occasion to research it a little bit for myself too. The following works for flipping a texture vertically (rotating and flipping horizontally should be just a vertical flip). Someone might have a more elaborate solution for you, but maybe this is enough to get you started.

    The Compute shader code:

     #pragma kernel CSMain
    // Create a RenderTexture with enableRandomWrite flag and set it
    // with cs.SetTexture
    RWTexture2D<float4> Result;
    Texture2D<float4> ImageInput;
    float2 flip;
    [numthreads(8,8,1)]
    void CSMain (uint3 id : SV_DispatchThreadID)
    {
        flip = float2(512 , 1024) - id.xy ;
        Result[id.xy] = float4(ImageInput[flip].x, ImageInput[flip].y, ImageInput[flip].z, 1.0);
    }
    

    and called from any script:

    public void FlipImage()
    {
        int kernelHandle = shader.FindKernel("CSMain");
        RenderTexture tex = new RenderTexture(512, 1024, 24);
        tex.enableRandomWrite = true;
        tex.Create();
    
        shader.SetTexture(kernelHandle, "Result", tex);
        shader.SetTexture(kernelHandle, "ImageInput", myTexture);
        shader.Dispatch(kernelHandle, 512/8 , 1024 / 8, 1);
    
        RenderTexture.active = tex;
        result.ReadPixels(new Rect(0, 0, tex.width, tex.height), 0, 0);
        result.Apply();
    }
    

    This takes an input Texture2D, flips it in the shader, applies it to a RenderTexture and to a Texture2D, whatever you need. Note that the image sizes are hardcoded in my instance and should be replaced by whatever size you need. (for within the shader use shader.SetInt(); )