I am working on a demonstrator for a Machine Learning based application.
To make things simple, let's assume that :
I am working with two files, run_ml.py and video_stream_flask.py.
The run_ml.py file basically captures an image from the camera, does all the processing, and draws things on the original image.Then, the final image is put in a queue.
This queue is passed among other arguments to a subprocess running the main() of video_stream_flask.py.
The goal here is to get the image from the queue, and display it on a webpage. And in facts, the initial goal is only to be able to display the final image on a webpage.
This is basically the code of the video_stream_flask.py :
import subprocess
from subprocess import PIPE
from flask import Flask, Response, render_template
import cv2
import multiprocessing
import logging
VERBOSE_DEBUG = 0
if not VERBOSE_DEBUG:
log = logging.getLogger('werkzeug')
log.setLevel(logging.ERROR)
TARGET = None
queue=multiprocessing.Queue()
global image
image=cv2.imread("output.jpg")
def get_queue(queue):
try:
data = queue.get(block=False)
print("Data retrieved from queue")
return data
except Exception as e:
# print("except : ",e)
# print("Queue is empty, no data to retrieve.")
return None
def get_current_ip():
"""
Get current IP Address by testing IP route
"""
process_get_ip = subprocess.Popen(["ip route get 1.2.3.4 | awk '{print $7}'"], shell=True, stdout=PIPE)
output = process_get_ip.communicate()[0].decode("utf-8")
return output
app = Flask(__name__)
@app.route('/')
def index():
return render_template("index.html")
def gen():
while True:
global image
global queue
# image = cv2.imread("output.jpg")
image=get_queue(queue)
if image is not None:
image = cv2.flip(image, 1)
ret, jpeg = cv2.imencode('.jpg', image)
frame = jpeg.tobytes()
yield (b'--frame\r\n'
b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')
print("Frame sent")
@app.route('/video_feed')
def video_feed():
return Response(gen(),
mimetype='multipart/x-mixed-replace; boundary=frame')
def main(lan, target,QUEUE):
global TARGET
TARGET = target
global queue
queue=QUEUE
if bool(lan):
subprocess.Popen(["ifconfig eth0 10.0.0.20 netmask 255.255.255.0 up"], shell=True, stdout=PIPE)
print("\nCONNECT TO : 10.0.0.20:5000")
app.run(host="10.0.0.20", port=5000, threaded=True)
else:
if TARGET:
ip_address = get_current_ip()[:-2]
print("\nCONNECT TO : " + repr(ip_address)[1:-1] + ":5000")
app.run(host=ip_address, port=5000, threaded=True)
else:
print("\nCONNECT TO : localhost:5000")
app.run(host="localhost", port=5000, threaded=True) #, use_reloader=False
The problem I am facing, is the latency. I get instantly almost a ten second latency when viewing the result on my webpage.
I do not understand why.
I have a basic example running, where the image is captured from the camera, directly in the same file and same process, and it works perfectly.
But here, when I try to retrieve the image from the main process, it lags.
I tried other ways of doing so like saving the image in the filesystem and reading it again, with mutexes to make sure there are no conflicts.
When I run my application, I can see on the output that images are yeld as soon as they are ready :
Data retrieved from queue
Frame sent
Model Inference time: 926.544ms
Data put into queue successfully.
Data retrieved from queue
Frame sent
Model Inference time: 923.772ms
Data put into queue successfully.
Data retrieved from queue
Frame sent
Model Inference time: 914.210ms
Data put into queue successfully.
Data retrieved from queue
Frame sent
Would you have any idea where does the problem comes from ? Or how to do what I want to do, which means, how to stream an image generated in another script, and stream it only once when it is ready ?
Many thanks,
I realized that the problem was not coming from flask, but from the OpenCV capture.
Explanations about the root cause are available here : OpenCV VideoCapture lag due to the capture buffer