Search code examples
unity-game-engineopenglimage-processingshaderhlsl

What is the best way to smoothen a noisy image filter?

I am currently trying to recreate the watercolor effect of Instagram in Unity.

Instagram: https://i.sstatic.net/pJAo6.jpg

My approach: https://i.sstatic.net/Ngnn3.jpg

My approach is rather noisy. This is the main code which creates the effect:

float3 stepColor(float3 col){
     const float3 lumvals = float3(0.5,0.7,1.0);

    float3 hsv = rgb2hsv(col);
    if(hsv.z <= 0.33){
        hsv.z = lumvals.x;
    }
    else if(hsv.z <= 0.55){
        hsv.z = lumvals.y;
    }
    else{
        hsv.z = lumvals.z;
    }
    return hsv2rgb(hsv);
}

Which algorithm would be suitable here to denoise and smoothen the end result as Instagram is achieving it?

Solution

  • Watercolor filters use something called mean shift analysis to average out the image while preserving features. It is an iterative approach where you make clusters of pixels gravitate towards their mean value.

    Here is a java code example: https://imagej.nih.gov/ij/plugins/mean-shift.html

    Here is a paper which describes the watercolor effect and its components (including edge darkening): http://maverick.inria.fr/Publications/2006/BKTS06/watercolor.pdf

    There is a github project with CUDA and OpenCL implementations, but if you want to actually understand the algorithm, i'd refer you to this page which explains it quite neatly using python code:

    http://www.chioka.in/meanshift-algorithm-for-the-rest-of-us-python/

    Another option from the top of my head is to use a Sobel/Roberts cross filter to detect all the borders in the image, and then use the inverse of this value as a mask for a gaussian blur. It won't give you the same nice layering effect though.