I am using GStreamer to stream live video / audio from a Pi3B with a picam module and USB microphone. My end goal is to use the audio from the one USB microphone in both the live video / audio stream AND as the input to a python script. I understand that this can be done with the ALSA dsnoop plugin and have been able to demonstrate it with this /etc/asound.conf config:
pcm.myTest {
type dsnoop
ipc_key 2241234
slave {
pcm "hw:1,0"
channels 1
}
}
pcm.!default {
type asym
playback.pcm {
type plug
slave.pcm "hw:0,0"
}
capture.pcm {
type plug
slave.pcm "myTest"
}
}
The video / audio stream works perfectly using the following GStreamer settings, but i am unable to use the microphone in other applications (note the "hw:1,0"):
#!/bin/bash
gst-launch-1.0 -v rpicamsrc vflip=true hflip=false \
name=src preview=0 fullscreen=0 bitrate=10000000 \
annotation-mode=time annotation-text-size=20 \
! video/x-h264,width=960,height=540,framerate=24/1 \
! h264parse \
! rtph264pay config-interval=1 pt=96 \
! queue max-size-bytes=0 max-size-buffers=0 \
! udpsink host=192.168.1.101 port=5001 \
alsasrc device=hw:1,0 \
! audioconvert \
! audioresample \
! opusenc \
! rtpopuspay \
! queue max-size-bytes=0 max-size-buffers=0 \
! udpsink host=192.168.1.101 port=5002
The following (which uses dsnoop) causes an issue in the video stream which looks like some kind of synchronization problem where instead of a nice smooth 24 frames per second I get one frame every ~2-3 seconds. The audio continues to work well and im able to use the USB mic simultaneously in other applications.
#!/bin/bash
gst-launch-1.0 -v rpicamsrc vflip=true hflip=false \
name=src preview=0 fullscreen=0 bitrate=10000000 \
annotation-mode=time annotation-text-size=20 \
! video/x-h264,width=960,height=540,framerate=24/1 \
! h264parse \
! rtph264pay config-interval=1 pt=96 \
! queue max-size-bytes=0 max-size-buffers=0 \
! udpsink host=192.168.1.101 port=5001 \
alsasrc device=plug:myTest \
! audioconvert \
! audioresample \
! opusenc \
! rtpopuspay \
! queue max-size-bytes=0 max-size-buffers=0 \
! udpsink host=192.168.1.101 port=5002
I've tried a few things that I've found in some peripherally related forums to no avail and im feeling kinda stuck. Do any of you have any suggestions on getting a stream to play nicely with dsnoop so that I can avoid buying another microphone for this project?
Thank you!
For posterity, I received a great tip from the GStreamer developer forum.
Adding provide-clock=false to the alsasrc line did the trick! so the GStreamer call becomes:
#!/bin/bash
gst-launch-1.0 -v rpicamsrc vflip=true hflip=false \
name=src preview=0 fullscreen=0 bitrate=10000000 \
annotation-mode=time annotation-text-size=20 \
! video/x-h264,width=960,height=540,framerate=24/1 \
! h264parse \
! rtph264pay config-interval=1 pt=96 \
! queue max-size-bytes=0 max-size-buffers=0 \
! udpsink host=192.168.1.101 port=5001 \
alsasrc device=plug:myTest provide-clock=false\
! audioconvert \
! audioresample \
! opusenc \
! rtpopuspay \
! queue max-size-bytes=0 max-size-buffers=0 \
! udpsink host=192.168.1.101 port=5002
One minor side effect of this approach is that the audio is out of sync with the video by about 0.5 seconds. Im curious to know if there is a way to sync the two up a little better or if this just one of the inevitable tradeoffs when trying to use a dsnoop device with gstreamer?