Video cat ear filter too wiggly

Question

I am making a video filter that adds cat ears to you like a Snapchat lenses. I am using opencv4 and Dlib. Dlib takes care of detecting the face. The problem is because of detection coordinates changes a little every frame the filter is too wiggly.

I tried to change the place of ears every 2-3 frames but not much changed.

while True:
    _, frame = cap.read()
    ear_mask.fill(0)
    gray_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    faces = detector(frame)

    for face in faces:
        ear_width, ear_height, top_left, degree = generateNewEar(frame)
        frame = generateImage(frame, ear_width,
                              ear_height, top_left, degree)
    cv2.imshow("Frame", frame)

How do people usually handle this issue?

Answer 1

"The problem is, because of detection, the coordinates changes a little bit every frame so the filter is too wiggly."

I don't use Python or OpenCV so I can't show you an example code. Some advice you can try.

(1) Camera noise could be affecting the detection. Try a soft blur on the input to smooth the pixels. Test different blur levels.

(2) Try updating ear positions once every second, then fine tune to X times per sec.

(3) Compare ear position of current frame vs previous frame. If distance too small (1 or 2 pixels) then ignore and use old position.