Search code examples
pythonpython-3.xopencvopticalflow

How to find a threshold number in Optical Flow in Python OpenCV

Using optical flow, I am trying to detect the motion of a coil attached to a motor. When the motor starts, coil runs smoothly but sometimes it starts to vibrate. I need to detect this vibration. I am not sure if optical flow is the correct approach or not but when tested with a stable movement vs vibrations I can see some colors showing during vibrations. Attached are the images:

Stable (running smoothly) Vibration
enter image description here enter image description here

Doing optical flow on a complete video frame will not work so I have cropped the corners of the coil and you can see a lot of colors showing when it starts to vibrate because when it runs smoothly, there is not much motion visible but when it vibrates, motion is visible and thus it's picked up in the optical flow.

Now I am trying to find out some kind of threshold so that I can print when it starts vibrating, when it's low vibration, and when it's high vibrations.

Using below code:

import cv2
import numpy as np

cap = cv2.VideoCapture("Coil.mp4")
ret, frame1 = cap.read()
frame1 = frame1[284: 383, 498:516]
prvs = cv2.cvtColor(frame1, cv2.COLOR_BGR2GRAY)
hsv = np.zeros_like(frame1)
hsv[..., 1] = 255


while cap:
    ret, frame2 = cap.read()
    frame2 = frame2[284: 383, 498:516]
    next = cv2.cvtColor(frame2, cv2.COLOR_BGR2GRAY)
    flow = cv2.calcOpticalFlowFarneback(prvs, next, None, 0.5, 3, 15, 3, 5, 1.2, 0)
    mag, ang = cv2.cartToPolar(flow[..., 0], flow[..., 1])
    hsv[..., 0] = ang * 180 / np.pi / 2
    hsv[..., 2] = cv2.normalize(mag, None, 0, 255, cv2.NORM_MINMAX)
    rgb = cv2.cvtColor(hsv, cv2.COLOR_HSV2BGR)

    cv2.imshow('Optical Flow', rgb)
    k = cv2.waitKey(30) & 0xff
    if k == 27:
        break
    prvs = next

cap.release()
cv2.destroyAllWindows()

When there is a motion, optical flow shows it in form of color so I am guessing there has to be some way to find out threshold. What can I try next?

Solution

  • I think using average magnitude should work for thresholding vibration. The cv2.calcOpticalFlowFarneback() returns a vector which

    finds an optical flow for each prev pixel using the algorithm so that prev(y,x)∼next(y+flow(y,x),x+flow(y,x)[0])

    So when we move it to polar and get the magnitude and angle, angle shows direction of the flow and the magnitude shows how much the pixel has moved.

    The bellow code is averaging the magnitude on each frame and plot it by time so you can see how average magnitude changes by the coil vibration.

    import cv2
import numpy as np
import matplotlib.pyplot as plt

cap = cv2.VideoCapture("Coil.mp4")
fps = cap.get(cv2.CAP_PROP_FPS)
ret, frame1 = cap.read()
frame1 = frame1[284: 383, 498:516]
prvs = cv2.cvtColor(frame1, cv2.COLOR_BGR2GRAY)

mags = []
while True:
    ret, frame2 = cap.read()
    if not ret:
        break
    frame2 = frame2[284: 383, 498:516]
    next = cv2.cvtColor(frame2, cv2.COLOR_BGR2GRAY)
    flow = cv2.calcOpticalFlowFarneback(prvs, next, None, 0.5, 3, 15, 3, 5, 1.2, 0)
    mag, ang = cv2.cartToPolar(flow[..., 0], flow[..., 1])
    mag_mean = cv2.mean(mag)
    mags.append(mag_mean)
    prvs = next

mags_arr = np.array(mags)
time_arr = np.arange(len(mags)) / fps
plt.plot(time_arr, mags_arr)
plt.savefig("vibrate-time.png")

    At the end you have vibration / time graph and you can choose the threshold for high vibration and low vibration.

    This is the plot for a 12 sec sample video I've tested.

    enter image description here