How to identify new faces with OpenIMAJ

Question

I've got a live stream of JPG images coming in on a different thread in my Java app and I want to continually scan for faces to output later a list of all the different faces that passed the camera while it was running, and how many times each face was seen. Here's my current code:

void doImageProcessing() {

    // Create face stuff
    FKEFaceDetector faceDetector = new FKEFaceDetector(new HaarCascadeDetector());
    EigenFaceRecogniser<KEDetectedFace, Person> faceRecognizer = EigenFaceRecogniser.create(512, new RotateScaleAligner(), 512, DoubleFVComparison.CORRELATION, Float.MAX_VALUE);
    FaceRecognitionEngine<KEDetectedFace, Extractor<KEDetectedFace>, Person> faceEngine = FaceRecognitionEngine.create(faceDetector, faceRecognizer);

    // Start loop
    while (true) {

        // Get next frame
        byte[] imgData = nextProcessingData;
        nextProcessingData = null;

        // Decode image
        BufferedImage img = ImageIO.read(new ByteArrayInputStream(imgData));

        // Detect faces
        FImage fimg = ImageUtilities.createFImage(img);
        List<KEDetectedFace> faces = faceEngine.getDetector().detectFaces(fimg);

        // Go through detected faces
        for (KEDetectedFace face : faces) {

            // Find existing person for this face
            Person person = null;
            try {

                List<IndependentPair<KEDetectedFace, ScoredAnnotation<Person>>> rfaces = faceEngine.recogniseBest(face.getFacePatch());
                ScoredAnnotation<Person> score = rfaces.get(0).getSecondObject();
                if (score != null)
                    person = score.annotation;

            } catch (Exception e) {
            }

            // If not found, create
            if (person == null) {

                // Create person
                person = new Person();
                System.out.println("Identified new person: " + person.getIdentifier());

                // Train engine to recognize this new person
                faceEngine.train(person, face.getFacePatch());

            } else {

                // This person has been detected before
                System.out.println("Identified existing person: " + person.getIdentifier());

            }

        }

    }

}

The problem is it always detects a face as new, even if it's the same face that was detected in the previous frame. rfaces is always empty. It can never identify an existing face. What am I doing wrong?

Also, I have no idea what the parameters for the EigenFaceRecognizer creator function should be, maybe that's why it's not recognizing anything...

Answer 1

The parameters you've given to the EigenFaceRecogniser.create() function are way off, so that's probably the likely cause of your problems. The following is more likely to work:

EigenFaceRecogniser<KEDetectedFace, Person> faceRecognizer = EigenFaceRecogniser.create(20, new RotateScaleAligner(), 1, DoubleFVComparison.CORRELATION, 0.9f);

Explaination:

• The first parameter is the number of principal components in the EigenFace algorithm; the exact value is normally determined experimentally, but something around ~20 is probably fine.

• The third parameter is the number of nearest neighbours to use for the KNN classifier. 1 nearest-neighbour should be fine.

• The final parameter is a distance threshold for the classifier. The correlation comparison returns a similarity measure (high values mean more similar), so the threshold given is a lower limit that must be exceeded. As we've set 1 nearest neighbour, then the distance between the most similar face and the query face must be greater than 0.9. Note that a value of 0.9 is just a guess; in order to optimise the performance of your recogniser you'll need to play around with this.

Another minor point - instead of:

    BufferedImage img = ImageIO.read(new ByteArrayInputStream(imgData));
    FImage fimg = ImageUtilities.createFImage(img);

It's generally better to let OpenIMAJ read your image as it works around a number of known problems with ImageIO's handling of certain types of JPEG:

    FImage fimg = ImageUtilities.readF(new ByteArrayInputStream(imgData));