Cannot instantiate OpenCV-4.0.1 CascadeClassifier in a Grails 2.4.4 Project

I want to crop 1x1 employee pictures from their upload file to be set as their avatar on my Grails application. I heard OpenCV do the job well so I used it inside my ImageService. The problem is it seems that it cannot find (or read) the CascadeClassifier XML file it needs:

class ImageService {
    final String FRONTAL_FACE_XML = "D:\\Devtools\\opencv\\build\\etc\\lbpcascades\\lbpcascade_frontalface_improved.xml"
    final String ORIGINAL_PICTURE = "D:\\Projects\\opencv\\grails-app\\assets\\4fc30smaegvq0z3mvgm9yhf6vtv9kv8bgryi9x08wuada8jxu3.jpg"
    final String CROPPED_PICTURE = "D:\\Projects\\opencv\\grails-app\\assets\\4fc30smaegvq0z3mvgm9yhf6vtv9kv8bgryi9x08wuada8jxu3_100.jpg"

    void opencvtest() {
        // Before I placed the OpenCV dll in the environment path, this line causes an error.
        System.loadLibrary(Core.NATIVE_LIBRARY_NAME)

        // UnsatisfiedLinkError here
        CascadeClassifier faceDetector = new CascadeClassifier(this.getClass().getResource("lbpcascade_frontalface_improved.xml").getPath());

        // Same error as well.
        // File cascadeFile = new File(FRONTAL_FACE_XML);
        // CascadeClassifier faceDetector = new CascadeClassifier(cascadeFile.getAbsolutePath());

        // And also here.
        // CascadeClassifier faceDetector = new CascadeClassifier(FRONTAL_FACE_XML);

        Mat image = HighGui.imread(ORIGINAL_PICTURE)
        faceDetector.detectMultiScale(image, face_Detections)

        Rect rect_Crop = null

        for (Rect rect : face_Detections.toArray()) {
            Core.rectangle(image, new Point(rect.x, rect.y), new Point(rect.x + rect.width, rect.y + rect.height), new Scalar(0, 255, 0))
            rectCrop = new Rect(rect.x, rect.y, rect.width, rect.height)
        }

        Mat image_roi = new Mat(image, rectCrop)
        HighGui.imwrite(CROPPED_PICTURE, image_roi)

        return
    }
}

And causes the following error:

java.lang.UnsatisfiedLinkError org.opencv.objdetect.CascadeClassifier.CascadeClassifier_0(Ljava/lang/String;)J

Environment: Windows 7, Java 1.8, Grails 2.4.4

Things I have done:

I have installed OpenCV 4.0.1 from their website: https://opencv.org/releases.html and unpack their distribution on D:\opencv\4.0.1.
I have included this on my environment path: D:\opencv\4.0.1\build\java\x64
Since I cannot find an official Maven dependency link, I copied the jar I found on D:\opencv\4.0.1\build\java into my grails lib directory.
I already tried copying all the dll and xml to C:\Windows\system32.
Thinking that the jar on the distribution might be "faulty", I replace it with this package compile "org.bytedeco.javacpp-presets:opencv:4.0.1-1.4.4" on through BuildConfig.groovy, still both cause errors on the same line.
I verified whether FRONTAL_FACE_XML and ORIGINAL_PICTURE are both correct file path, and they are.

Solution

We will be using OpenCV using VC15 on Windows for this example. I have yet to learn how to port this on Linux.

Legend:

APPLICATION_PATH = D:\application
JAVA_DLL_PATH = D:\opencv\4.0.1\build\java\x64
VC_DLL_PATH = D:\opencv\4.0.1\build\x64\vc15\bin

Optional:

Go to %VC_DLL_PATH% directory.
Create a directory named debug (or whatever you prefer) and move all *d.dll files there. These files that ends with *d.dll might cause an error when you run them since it is looking for vc15 debug dlls that are not part of the basic Visual C++ 2015.

Configuration:

Create a directory in %APPLICATION_PATH%\src directory named files (or whichever you prefer). It should be similar to this.

%APPLICATION_PATH% ├── bin\ ├── grails-app\ ├── lib\ ├── ... ├── src\ │ ├── groovy\ │ ├── java\ │ └── files\ --new directory └── ...
1. Copy all *.dlls from %JAVA_DLL_PATH% and %VC_DLL_PATH% directories to the newly created directory %APPLICATION_PATH%\src\files.
2. Include the newly created directory to your .classpath file.
<?xml version="1.0" encoding="UTF-8"?> <classpath> <classpathentry excluding="spring/" kind="src" path="grails-app/conf"/> ... <classpathentry kind="src" path="src/files"/> ... </classpath>
1. Create another directory in your %APPLICATION_PATH% named files (or whichever you prefer).
%APPLICATION_PATH% ├── bin\ ├── grails-app\ ├── files\ --new directory └── ...
1. Copy here the CascadeClassifiers xml files that you will use. They are found in \opencv\4.0.1\build\etc directory.
2. Even after copying the dlls inside %APPLICATION_PATH%, we still need to include them on the system path. Edit your environment variables and include %JAVA_DLL_PATH% and %VC_DLL_PATH% directories to PATH.
3. Also, even if we copied the dlls and included the library directories into the system path, we still need to include it to the Tomcat JVM library path. Open your BuildConfig.groovy and add this:
grails.tomcat.jvmArgs = ["-Djava.library.path=D:\opencv\4.0.1\build\x64\vc15\bin;D:\opencv\4.0.1\build\java\x64"]
1. Include the directories to your Config.groovy.
openCV { cascadeClassifiers = "D:\\application\\files\\opencv" home = "D:\\opencv\\4.0.1\\build\\x64\\vc15\\bin" java = "D:\\opencv\\4.0.1\\build\\java\\x64" }

How to use:

You need to create a *.java file instead of a *.groovy file in %APPLICATION_PATH%\src\java. I have yet to check why it doesn't work on .groovy file.

import grails.util.Holders;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.UUID;
import org.apache.commons.io.FilenameUtils;
import org.apache.commons.io.FileUtils;
import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.core.MatOfRect;
import org.opencv.core.Rect;
import org.opencv.imgcodecs.Imgcodecs;
import org.opencv.objdetect.CascadeClassifier;

public class ImageService {
    static final String[] OPENCV_SUPPORTED_EXTENSION = new String[]{"bmp", "dib", "jp2", "jpe", "jpeg", "jpg", "pbm", "pgm", "png", "ppm", "ras", "sr", "tif", "tiff"};

    public static File cropImage(final File originalFile) {
        loadLibrary();
        return cropImage(originalFile, originalFile.getParentFile(), getCascadeClassifiers());
    }

    public static File cropImage(final File originalFile, final ArrayList<CascadeClassifier> cascadeClassifiers) {
        return cropImage(originalFile, originalFile.getParentFile(), cascadeClassifiers);
    }

    public static File cropImage(final File originalFile, final File targetDirectory) {
        loadLibrary();
        return cropImage(originalFile, originalFile.getParentFile(), getCascadeClassifiers());
    }

    public static File cropImage(final File originalFile, final File targetDirectory, final ArrayList<CascadeClassifier> cascadeClassifiers) {
        ArrayList<File> siblingFiles = getFaces(originalFile, cascadeClassifiers);
        File maxFile = null;
        int maxWidth = 0;

        for(int x = 0; x < siblingFiles.size(); x++) {
            Mat image = Imgcodecs.imread(siblingFiles.get(x).getAbsolutePath(), Imgcodecs.IMREAD_UNCHANGED);

            if(image.width() > maxWidth) {
                maxFile = siblingFiles.get(x);
                maxWidth = image.width();
            }
        }

        File croppedFile = null;
        if(maxFile != null) {
            croppedFile = new File(targetDirectory.getAbsolutePath() +
                File.separator +
                originalFile.getName());

            try {
                FileUtils.copyFile(maxFile, croppedFile);
            }
            catch(IOException e) {}
        }

        for(int y = 0; y < siblingFiles.size(); y++) {
            siblingFiles.get(y).delete();
        }

        System.gc();
        System.runFinalization();

        return croppedFile;
    }

    public static ArrayList<CascadeClassifier> getCascadeClassifiers() {
        ArrayList<CascadeClassifier> classifiers = new ArrayList<CascadeClassifier>();

        final String[] cascadeSupportedExtensions = new String[]{"xml"};
        final String cascadeClassifierPath = Holders.getFlatConfig().get("openCV.cascadeClassifiers").toString();
        File cascadeClassifierDirectory = new File(cascadeClassifierPath);
        ArrayList<File> detectors = new ArrayList<File>(FileUtils.listFiles(cascadeClassifierDirectory, cascadeSupportedExtensions, false));

        for(int y = 0; y < detectors.size(); y++) {
            CascadeClassifier faceDetector = new CascadeClassifier();
            faceDetector.load(detectors.get(y).getAbsolutePath());

            classifiers.add(faceDetector);
        }

        return classifiers;
    }

    public static ArrayList<File> getFaces(final File originalFile) {
        loadLibrary();
        return getFaces(originalFile, getCascadeClassifiers());
    }

    public static ArrayList<File> getFaces(final File originalFile, final ArrayList<CascadeClassifier> cascadeClassifiers) {
        File temporaryFile = new File(originalFile.getParentFile().getAbsolutePath() +
            File.separator +
            UUID.randomUUID().toString() +
            "." + FilenameUtils.getExtension(originalFile.getName()));

        try {
            FileUtils.copyFile(originalFile, temporaryFile);
        }
        catch(IOException e) {}

        final int frame = 9;
        final int offset = 8;
        int rotateCounter = 0;

        Integer marginX, marginY, marginWidth, marginHeight;
        Integer pxPerOffset, excess;

        ArrayList<File> siblingFiles = new ArrayList<File>();

        while(rotateCounter < 4) {
            if(rotateCounter > 0) {
                Mat image = Imgcodecs.imread(temporaryFile.getAbsolutePath(), Imgcodecs.IMREAD_UNCHANGED);

                Core.transpose(image, image);
                Core.flip(image, image, 1);

                Imgcodecs.imwrite(temporaryFile.getAbsolutePath(), image);

                image.release();
            }

            for(int y = 0; y < cascadeClassifiers.size(); y++) {
                CascadeClassifier faceDetector = cascadeClassifiers.get(y);
                Mat image = Imgcodecs.imread(temporaryFile.getAbsolutePath(), Imgcodecs.IMREAD_UNCHANGED);

                MatOfRect faceDetections = new MatOfRect();
                faceDetector.detectMultiScale(image, faceDetections);

                Rect defaultRect = null;
                Rect marginRect = null;
                Rect[] facesRect = faceDetections.toArray();

                for(int z = 0; z < facesRect.length; z++) {
                    defaultRect = facesRect[z];
                    pxPerOffset = defaultRect.width / frame;

                    marginX = defaultRect.x - (pxPerOffset * offset);
                    marginY = defaultRect.y - (pxPerOffset * offset);

                    marginWidth = defaultRect.width + (offset * pxPerOffset * 2);
                    marginHeight = defaultRect.height + (offset * pxPerOffset * 2);

                    excess = Math.max(
                        0 - marginX,
                            Math.max(0 - marginY,
                                Math.max(marginX + marginWidth - image.width(),
                                    Math.max(marginY + marginHeight - image.height(), 0)))
                    );

                    if(excess > 0) {
                        marginX += excess;
                        marginY += excess;

                        marginWidth -= excess * 2;
                        marginHeight -= excess * 2;
                    }

                    marginRect = new Rect(marginX, marginY, marginWidth, marginHeight);
                    Mat imageWithMargin = new Mat(image, marginRect);

                    String croppedFileName = temporaryFile.getParentFile().getAbsolutePath() +
                        File.separator +
                        UUID.randomUUID().toString() + "_" +
                        y + "_" +
                        rotateCounter + "_" +
                        z + "." +
                        FilenameUtils.getExtension(temporaryFile.getName());

                    Imgcodecs.imwrite(croppedFileName, imageWithMargin);

                    siblingFiles.add(new File(croppedFileName));

                    imageWithMargin.release();
                }

                image.release();
            }

            rotateCounter++;
        }

        temporaryFile.delete();

        return siblingFiles;
    }

    public static void loadLibrary() {
        System.loadLibrary(Core.NATIVE_LIBRARY_NAME);

        final String[] urls = new String[]{
            Holders.getFlatConfig().get("openCV.java").toString(),
            Holders.getFlatConfig().get("openCV.home").toString()
        };

        final String[] validExtensions = new String[]{"dll"};
        ArrayList<File> dlls = new ArrayList<File>();
        for(int y = 0; y < urls.length; y++) {
            dlls.addAll(FileUtils.listFiles(new File(urls[y]), validExtensions, false));
        }

        for(int y = 0; y < dlls.size(); y++) {
            for(int z = 0; z < validExtensions.length; z++) {
                System.loadLibrary(dlls.get(y).getName().replace("." + validExtensions[z], ""));
                System.load(dlls.get(y).getAbsolutePath());
            }
        }
    }
}

Then we use it to our groovy service classes.

class TestService {
    def openCVTest() {
        File picture = new File("D:\\original.jpg");
        File savingDirectory = new File("D:\\");

        ImageService.cropImage(picture, savingDirectory);

        return;
    }
}