In my project I'm making a drone with a raspberry pi. I need to stream video from my raspberry pi camera with as low latency as possible and share that stream to multiple clients. I achieved a simple stream basing on the code :
import io
import picamera
import logging
import socketserver
from threading import Condition
from http import server
PAGE="""\
<html>
<head>
<title>picamera MJPEG streaming</title>
</head>
<body>
<h1>PiCamera MJPEG Streaming</h1>
<img src="stream.mjpg" width="640" height="480" />
</body>
</html>
"""
class StreamingOutput(object):
def __init__(self):
self.frame = None
self.buffer = io.BytesIO()
self.condition = Condition()
def write(self, buf):
if buf.startswith(b'\xff\xd8'):
# New frame, copy the existing buffer's content and notify all
# clients it's available
self.buffer.truncate()
with self.condition:
self.frame = self.buffer.getvalue()
self.condition.notify_all()
self.buffer.seek(0)
return self.buffer.write(buf)
class StreamingHandler(server.BaseHTTPRequestHandler):
def do_GET(self):
if self.path == '/':
self.send_response(301)
self.send_header('Location', '/index.html')
self.end_headers()
elif self.path == '/index.html':
content = PAGE.encode('utf-8')
self.send_response(200)
self.send_header('Content-Type', 'text/html')
self.send_header('Content-Length', len(content))
self.end_headers()
self.wfile.write(content)
elif self.path == '/stream.mjpg':
self.send_response(200)
self.send_header('Age', 0)
self.send_header('Cache-Control', 'no-cache, private')
self.send_header('Pragma', 'no-cache')
self.send_header('Content-Type', 'multipart/x-mixed-replace; boundary=FRAME')
self.end_headers()
try:
while True:
with output.condition:
output.condition.wait()
frame = output.frame
self.wfile.write(b'--FRAME\r\n')
self.send_header('Content-Type', 'image/jpeg')
self.send_header('Content-Length', len(frame))
self.end_headers()
self.wfile.write(frame)
self.wfile.write(b'\r\n')
except Exception as e:
logging.warning(
'Removed streaming client %s: %s',
self.client_address, str(e))
else:
self.send_error(404)
self.end_headers()
class StreamingServer(socketserver.ThreadingMixIn, server.HTTPServer):
allow_reuse_address = True
daemon_threads = True
with picamera.PiCamera(resolution='640x480', framerate=24) as camera:
output = StreamingOutput()
camera.start_recording(output, format='mjpeg')
try:
address = ('', 8000)
server = StreamingServer(address, StreamingHandler)
server.serve_forever()
finally:
camera.stop_recording()
The above code starts a streaming server on the raspberry pi and provides a view from picamera. The latency is around 200ms and that is really good for me, but when more clients connect the latency is increasing. With 5 users, there is already quite a lot of lag. This is the main problem in my project, because I have to stream in real time for all users. I predict 20-30 users. I'm just trying achive similar effect to youtube livestreams or google meets where multiple clients can connect and watch view from my webcam but with the least latency.
So I thought about creating a second server that would be only receive one stream from my picamera and then would restream it to multiple clients without much delay. I tried to make a similar server to the one with raspberry pi based on the above code, but with the opencv. The picamera stream was intercepted with opencv and displayed, but unfortunately the latency was quite high. I tried also streaming to youtube via rtmp protocol with raspivid and to datarhei restreamer on docker via rtsp protocol with gstreamer, but the latency is around 2-3s and that is too much for me. Is there any way to do that or maybe other solutions? I spent a lot of time for that and I have no idea how to do that... Any advice would be helpful, because I'm newbie in video streaming. PS Sorry for my unperfect english
Video streaming often ends up being a balance between bandwidth, latency and quality.
Most online movie and live entertainment, sports etc streaming services use HSL or DASH streaming to provide the quality they need and will have a much higher latency that you are aiming for.
Similarly, serving many individual clients on a small PI device is likely to overrun the devices capacity as you have seen.
I think your idea of having the drone send the stream to a server which then provides streams to clients is the most viable approach, but you probably want to look at a real time streaming protocol based server.
WebRTC is the most obvious current choice, I think, for this type of streaming - it is specifically designed to prioritise latency over other factors, like quality, to enable real time voice and video communications.
Open source webRTC server exists which would be good starting point - e.g.:
Paid solutions also exists, like the enterprise edition of Ant Media WebRTC steaming server that lists IP Camera streaming amongst its usage scenarios:
There is also a good overview of a node.js based WebRTC solution here which reportedly achieves 400ms latency if that meets your needs: https://stackoverflow.com/a/67887822/334402