Custom filter bank is not generating the expected output

Question

Please, refer to this article.

I have implemented the section 4.1 (Pre-processing).

The preprocessing step aims to enhance image features along a set of chosen directions. First, image is grey-scaled and filtered with a sharpening filter (we subtract from the image its local-mean filtered version), thus eliminating the DC component.

We selected 12 not overlapping filters, to analyze 12 different directions, rotated with respect to 15° each other.

GitHub Repositiry is here.

Since, the given formula in the article is incorrect, I have tried two sets of different formulas.

The first set of formula,

The second set of formula,

The expected output should be,

Neither of them are giving proper results.

Can anyone suggest me any modification?

---

GitHub Repository is here.

Most relevalt part of the source code is here:

    public List<Bitmap> Apply(Bitmap bitmap)
    {
        Kernels = new List<KassWitkinKernel>();

        double degrees = FilterAngle;

        KassWitkinKernel kernel;
        for (int i = 0; i < NoOfFilters; i++)
        {
            kernel = new KassWitkinKernel();
            kernel.Width = KernelDimension;
            kernel.Height = KernelDimension;
            kernel.CenterX = (kernel.Width) / 2;
            kernel.CenterY = (kernel.Height) / 2;
            kernel.Du = 2;
            kernel.Dv = 2;
            kernel.ThetaInRadian = Tools.DegreeToRadian(degrees);
            kernel.Compute();

            //SleuthEye
            kernel.Pad(kernel.Width, kernel.Height, WidthWithPadding, HeightWithPadding);

            Kernels.Add(kernel);

            degrees += degrees;
        }

        List<Bitmap> list = new List<Bitmap>();

        Bitmap image = (Bitmap)bitmap.Clone();

        //PictureBoxForm f = new PictureBoxForm(image);
        //f.ShowDialog();

        Complex[,] cImagePadded = ImageDataConverter.ToComplex(image);

        Complex[,] fftImage = FourierTransform.ForwardFFT(cImagePadded);

        foreach (KassWitkinKernel k in Kernels)
        {
            Complex[,] cKernelPadded = k.ToComplexPadded();
            Complex[,] convolved = Convolution.ConvolveInFrequencyDomain(fftImage, cKernelPadded);

            Bitmap temp = ImageDataConverter.ToBitmap(convolved);



            list.Add(temp);
        }

        return list;
    }

Answer 1

Perhaps the first thing that should be mentioned is that the filters should be generated with angles which should increase in FilterAngle (in your case 15 degrees) increments. This can be accomplished by modifying KassWitkinFilterBank.Apply as follow (see this commit):

public List<Bitmap> Apply(Bitmap bitmap)
{
    // ...

    // The generated template filter from the equations gives a line at 45 degrees. 
    // To get the filter to highlight lines starting with an angle of 90 degrees
    // we should start with an additional 45 degrees offset.
    double degrees = 45;

    KassWitkinKernel kernel;
    for (int i = 0; i < NoOfFilters; i++)
    {
        // ... setup filter (unchanged)

        // Now increment the angle by FilterAngle
        // (not "+= degrees" which doubles the value at each step)
        degrees += FilterAngle;
    }

This should give you the following result:

It is not quite the result from the paper and the differences between the images are still quite subtle, but you should be able to notice that the scratch line is most intense in the 8th figure (as would be expected since the scratch angle is approximately 100-105 degrees).

To improve the result, we should feed the filters with a pre-processed image in the same way as described in the paper:

First, image is grey-scaled and filtered with a sharpening filter (we subtract from the image its local-mean filtered version), thus eliminating the DC component

When you do so, you will get a matrix of values, some of which will be negative. As a result this intermediate processing result is not suitable to be stored as a Bitmap. As a general rule when performing image processing, you should keep all intermediate results in double or Complex as appropriate, and only convert back the final result to Bitmap for visualization.

Integrating your changes to add image sharpening from your GitHub repository while keeping intermediate results as doubles can be achieve by changing the input bitmap and temporary image variables to use double[,] datatype instead of Bitmap in the KassWitkinFilterBank.Apply method (see this commit):

public List<Bitmap> Apply(double[,] bitmap)
{
    // [...]

    double[,] image = (double[,])bitmap.Clone();

    // [...]
}

which should give you the following result:

Or to better highlight the difference, here is figure 1 (0 degrees) on the left, next to figure 8 (105 degrees) on the right: