Python OpenCV Strange result for image entropy on video

Question

I am working on a project that aims to extract the "interesting" sequences from an mp4 video using different concepts.

One of them is supposed to be the image entropy and right now, I am a bit stuck.

I have followed this tutorial to get the entropy for a few selected screenshots from the video: https://scikit-image.org/docs/dev/auto_examples/filters/plot_entropy.html

For that, I got results like this one which is what I wanted.

To apply it to my test video, I did the following:

import cv2
from skimage.filters.rank import entropy
from skimage.morphology import disk

# Creating a VideoCapture object to read the video
video = cv2.VideoCapture('mypath/samplevideo.mp4')

if not video.isOpened():
    print("Error reading video file")

width = int(video.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(video.get(cv2.CAP_PROP_FRAME_HEIGHT))
size = (width, height)
fourcc = cv2.VideoWriter_fourcc(*'XVID')

result = cv2.VideoWriter('filename.avi', cv2.VideoWriter_fourcc(*'VIDX'), 30, size)
# Loop until the end of the video
while video.isOpened():
    # Capture frame-by-frame
    ret, frame = video.read()

    # Display the resulting frame
    cv2.imshow('Frame', frame)

    # apply the entropy to each frame
    img = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    entropy_frame = entropy(img, disk(5))
    cv2.imshow('Entropy', entropy_frame)

    result.write(entropy_frame)
    # define q as the exit button
    if cv2.waitKey(25) & 0xFF == ord('q'):
        break

# release the video capture object
video.release()
result.release()
# Closes all the windows currently opened.
cv2.destroyAllWindows()

This code yields two windows, one with the original video (as wanted) and one that is supposed to show the same video but with the image entropy applied. At the end, it is supposed to save the entropy video. What I get right now in the "entropy window" is something like this (left is the original video, right the entropy) which does not at all look like the result I want.

What can I fix to get the result that I want? Thanks in advance

Answer 1

You are getting strange results because the range of entropy_frame is about [0, 6.0].
You need to convert the range to [0, 255] and cast the result from float to uint8.

A simple way to do the range conversion is multiply by (255/max(entropy_mat)):

entropy_frame = (entropy_mat * 255/np.max(entropy_mat)).astype(np.uint8)

Note:
The example uses matplotlib method ax1.imshow, that automatically performs linear stretching for display, so it displayed correctly (but it can't be used as a video frame).

---

Here is a single frame example:

import numpy as np
import cv2
from skimage.filters.rank import entropy
from skimage.morphology import disk

frame = cv2.imread('frame.png')

# Display the resulting frame
cv2.imshow('Frame', frame)

# apply the entropy to each frame
img = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
entropy_mat = entropy(img, disk(5))  # Range is about [0, 6]

# Apply linear stretching - lower 1% goes to 0, and maximum 1% goes to 255
# lo_val, up_val = np.percentile(entropy_mat, (1, 99))  # Get the value of lower and upper 1% of all pixels [0, 5.48]
# https://stackoverflow.com/questions/49656244/fast-imadjust-in-opencv-and-python
# entropy_frame = np.clip((entropy_mat - lo_val)*(255/(up_val - lo_val)), 0, 255).astype(np.uint8)

# Or simply multiply by 255/np.max(entropy_mat)
entropy_frame = (entropy_mat * 255/np.max(entropy_mat)).astype(np.uint8)

cv2.imshow('Entropy', entropy_frame)

cv2.imwrite('entropy_frame.png', entropy_frame)

cv2.waitKey()
cv2.destroyAllWindows()

---

Notes:

• Modify cv2.VideoWriter_fourcc(*'VIDX') to cv2.VideoWriter_fourcc(*'XVID') ('VIDX' displays an error message).

• entropy_frame is a Grayscale image.
  you need to set VideoWriter isColor argument to False:

   result = cv2.VideoWriter('filename.avi', cv2.VideoWriter_fourcc(*'VIDX'), 30, size, isColor=False)
  

• You need to break the loop if ret is False:

   if not ret:
       break
  

---

Result:

---

Complete code:

import numpy as np
import cv2
from skimage.filters.rank import entropy
from skimage.morphology import disk

# Creating a VideoCapture object to read the video
video = cv2.VideoCapture('mypath/samplevideo.mp4')

if not video.isOpened():
    print("Error reading video file")

width = int(video.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(video.get(cv2.CAP_PROP_FRAME_HEIGHT))
size = (width, height)
fourcc = cv2.VideoWriter_fourcc(*'XVID')

result = cv2.VideoWriter('filename.avi', cv2.VideoWriter_fourcc(*'XVID'), 30, size, isColor=False)
# Loop until the end of the video
while video.isOpened():
    # Capture frame-by-frame
    ret, frame = video.read()

    if not ret:
        break  # Break the loop if ret is false (last video frame).

    # Display the resulting frame
    cv2.imshow('Frame', frame)

    # apply the entropy to each frame
    img = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    entropy_mat = entropy(img, disk(5))
    entropy_frame = (entropy_mat * 255/np.max(entropy_mat)).astype(np.uint8)
    cv2.imshow('Entropy', entropy_frame)

    result.write(entropy_frame)
    # define q as the exit button
    if cv2.waitKey(25) & 0xFF == ord('q'):
        break

# release the video capture object
video.release()
result.release()
# Closes all the windows currently opened.
cv2.destroyAllWindows()