OpenCV with Python - Placing an image of a hat over the head of a webcam feed

I am trying to place a png image of a hat over the head of a webcam feed. I am trying to detect a face and place the image above it. This is my code so far -

import cv2
import numpy as np

face_cascade = cv2.CascadeClassifier('haarcascades/haarcascade_frontalface_default.xml')

imghat = cv2.imread('hat.png', -1)

print imghat is None

imghatGray = cv2.cvtColor(imghat, cv2.COLOR_BGR2GRAY)

ret, orig_mask = cv2.threshold(imghatGray, 0, 255, cv2.THRESH_BINARY)
orig_mask_inv = cv2.bitwise_not(orig_mask)

# Convert hat image to BGR
# and save the original image size (used later when re-sizing the image)
imghat = imghat[:,:,0:3]
origHatHeight, origHatWidth = imghat.shape[:2]

video_capture = cv2.VideoCapture(0)

while True:

    ret, frame = video_capture.read()

    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    faces = face_cascade.detectMultiScale(gray, 1.3, 5, flags=cv2.cv.CV_HAAR_SCALE_IMAGE)

    for (x, y, w, h) in faces:
        print "x : %d , y : %d, w: %d, h: %d " %(x,y,w,h)
        cv2.rectangle(frame, (x,y), (x+w, y+h), (255,0,0), 2)
        cv2.rectangle(frame, (x-15,y-h), (x+w+15, y), (255,255,0), 2)

        print w
        print h
        hatWidth = w
        hatHeight = hatWidth * origHatHeight / origHatWidth

        roi_gray = gray[y-hatHeight:y, x-15:x+w+15]
        roi_color = frame[y-hatHeight:y, x-15:x+w+15]

        # Center the hat
        x1 = x - 15
        y1 = y - h
        x2 = x + w +15
        y2 = y

        cv2.rectangle(frame, (x1,y1), (x2, y2), (0,255,0), 2)

        # Check for clipping
        if x1 < 0:
            x1 = 0
        if y1 < 0:
            y1 = 0
        if x2 > w:
            x2 = w
        if y2 > h:
            y2 = h

        # Re-calculate the width and height of the hat image
        hatWidth = x2 - x1
        hatHeight = y2 - y1

        # Re-size the original image and the masks to the hat sizes
        # calcualted above
        hat = cv2.resize(imghat, (hatWidth,hatHeight), interpolation = cv2.INTER_AREA)
        mask = cv2.resize(orig_mask, (hatWidth,hatHeight), interpolation = cv2.INTER_AREA)
        mask_inv = cv2.resize(orig_mask_inv, (hatWidth,hatHeight), interpolation = cv2.INTER_AREA)

        # take ROI for hat from background equal to size of hat image
        roi = roi_color[y1:y2, x1:x2]

        # roi_bg contains the original image only where the hat is not
        # in the region that is the size of the hat.
        roi_bg = cv2.bitwise_and(roi,roi,mask = mask_inv)

        # roi_fg contains the image of the hat only where the hat is
        roi_fg = cv2.bitwise_and(hat,hat,mask = mask)

        # join the roi_bg and roi_fg
        dst = cv2.add(roi_bg,roi_fg)

        # place the joined image, saved to dst back over the original image
        roi_color[y1:y2, x1:x2] = dst

        break


    # Display the resulting frame
        cv2.imshow('Video', frame)

        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

video_capture.release()
cv2.destroyAllWindows()

I get this error - OpenCV Error: Assertion failed (s >= 0) in setSize everytime I run it. The webcam start and closes abruptly. The error is somewhere in -

            hat = cv2.resize(imghat, (hatWidth,hatHeight), interpolation = cv2.INTER_AREA)
            mask = cv2.resize(orig_mask, (hatWidth,hatHeight), interpolation = cv2.INTER_AREA)
            mask_inv = cv2.resize(orig_mask_inv, (hatWidth,hatHeight), interpolation = cv2.INTER_AREA)

The values of hatWidth and hatHeight are negative. But I cannot find an error in the assignment of the coordinates. Is it because of the ROI in the program?

Solution

In the code Center the hat you have:

    x1 = x - 15
    x2 = x + w +15

where x1 and x2 seem to be the horizontal bounds of the hat.

Then, a few lines later, without x1 and x2 changing values you have

    # Check for clipping
    if x1 < 0:
        x1 = 0
    if x2 > w:
        x2 = w

This code will always modify x2 since x2 is by definition greater than w, in facts it's x + w + 15. This is probably not what you intended.

A few lines further you set hatWidth to

hatWidth = x2 - x1

At this point, x2 is always w due to the above 'clipping' code

So, if x2 is less than x1 this will make hatWidth negative which causes the problem you're seeing in cv2.resize( ... ).

When will x2 be less than x1? Well, x2 is w and x1 is x - 15 so whenever w < (x - 15) which is w + 15 < x i.e. whenever the position of the detected face is further right than the width of the face plus 15; this seems like something which could happen quite regularly and should't actually be an issue.

I suspect your clipping code should be checking for x2 being greater than the image width, not the face width.

You have a similar problem with clipping y2