How to save video recording through edge detection on pi

Question

I am currently trying to run my code on the raspberry pi using a raspberry camera. However, when I run it, the file saves, but it either does not allow me to view it on vlc or it plays but static just appears - depending on the codec I use.

I have tried multiple codecs such as XVID, MJPG, MPEG, H264, and only MJPG allows a playback, but it plays back as static. While I am recording, I can see where the camera is detecting the edges all around the room. However, it does not show back the way it recorded. I have tried converting the .avi to .mp4 which is of no help. I uploaded the file to youtube, and it played the same way. I have also ran a different code without the edge detection which always seems to work just fine and play back perfectly everytime. I will include it down below in addition to the code for the edge detection.

import numpy as np
import cv2

cap = cv2.VideoCapture(0)

# Define the codec and create VideoWriter object
# Remember, you might need to change the XVID codec to something else (MPEG?)
fourcc = cv2.VideoWriter_fourcc(*'MJPG')
out = cv2.VideoWriter('vid5.avi',fourcc, 20.0, (640,480), False)

while(cap.isOpened()):
    ret, frame = cap.read()
    if ret==True:
        frame = cv2.Canny(frame,100,200)

        # write the flipped frame
        out.write(frame)

        cv2.imshow('frame',frame)
        if cv2.waitKey(1) & 0xFF == ord('q'):
        break
    else:
        break

# Release everything if job is finished
cap.release()
out.release()
cv2.destroyAllWindows()

--------------------------------------------------------------------------
#the following is the code that plays back just fine without the canny.

import numpy as np
import cv2

cap = cv2.VideoCapture(0)

fourcc = cv2.VideoWriter_fourcc(*'MJPG')
out = cv2.VideoWriter('output.avi', fourcc, 20.0, (640,480))

while(cap.isOpened()):
    ret, frame = cap.read()
    if ret==True:
        frame = cv2.flip(frame,0)

        out.write(frame)

        cv2.imshow('frame',frame)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    else:
        break
cap.release()

out.release()

cv2.destroyAllWindows()

I expected to see the video playback through the edge detection applied, but it did not.

Answer 1

According to the OpenCV documentation (Link: https://docs.opencv.org/4.0.0/dd/d9e/classcv_1_1VideoWriter.html#ac3478f6257454209fa99249cc03a5c59):

The cv2.VideoWriter's flag isColor is only supported on Windows. On other platforms, the encoder is always expecting color frames (i.e channels == 3). In your above code, when you are trying to apply Canny Edge Detection on RGB frame, it is returning a grayscale frame image. Hence you are not able to see the playback.

Try converting the single channel frame output to three channel frame before passing it into VideoWriter:

frame = cv2.Canny(frame,100,200)       # Single channel. Shape = (640, 480)
frame = np.expand_dims(frame, axis=-1) # Single channel. Shape = (640, 480, 1)
frame = np.concatenate((frame, frame, frame), axis=2) # 3 channel. Shape = (640, 480, 3)

out.write(frame)