I need to demux and mux a stream with gstreamer (everything is tested with version 1.10.4 on CentOS and 1.14.1 on Ubuntu)
This means I have one input mpegts stream and need to output one stream with video/audio and one with klv metadata.
If my input streams consists of exactly those elements (video, audio and klv metadata stream) everything works as expected and I can send my video/audio and my klv metadata to two different udp sinks.
Now we are receiving a mpegts streams which has additional unknown binary streams within (we are not interested in those).
Here's the output of ffprobe
Working stream
Stream #0:0[0x100]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1920x1088, 25 fps, 25 tbr, 90k tbn, 50 tbc
Stream #0:1[0x101]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 128 kb/s
Stream #0:2[0x102]: Data: klv (KLVA / 0x41564C4B)
additional streams, not working
Stream #0:0[0x12c]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
Stream #0:1[0x12d]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 256 kb/s
Stream #0:2[0x5]: Data: bin_data ([6][0][0][0] / 0x0006)
Stream #0:3[0x262]: Data: bin_data (FBID / 0x44494246)
Stream #0:4[0x258]: Data: klv (KLVA / 0x41564C4B)
as you can see the two unknown streams come before the klv stream.
I have registered tsdemux callbacks pad-added and no-more-pads.
With the working stream gstreamer calls pad-added three times (once for every stream) and then no-more-pads once. Everything as I would expect.
With the non working stream gstreamer calls pad-added two times (for the first two streams) and then no-more-pads once.
It seems that tsdemux stops looking for streams/possible pads as soon as it encountered one stream it can't handle.
I would have expected that it will continue to look until there are no more streams available.
EDIT
I added the debug log from gstreamer
mpegtsbase mpegtsbase.c:532:mpegts_base_program_add_stream:[00m pid:0x0005, stream_type:0x006
mpegtsbase mpegtsbase.c:557:mpegts_base_program_add_stream:[00m PID 0x0005, registration_id 00000000 (....)
mpegtsbase mpegtsbase.c:291:mpegts_get_descriptor_from_stream:[00m Searching for tag 0x0a in stream 0x0005 (stream_type 0x06)
mpegtsbase mpegtsbase.c:291:mpegts_get_descriptor_from_stream:[00m Searching for tag 0x59 in stream 0x0005 (stream_type 0x06)
mpegtsbase mpegtsbase.c:291:mpegts_get_descriptor_from_stream:[00m Searching for tag 0x6a in stream 0x0005 (stream_type 0x06)
mpegtsbase mpegtsbase.c:291:mpegts_get_descriptor_from_stream:[00m Searching for tag 0x7a in stream 0x0005 (stream_type 0x06)
mpegtsbase mpegtsbase.c:291:mpegts_get_descriptor_from_stream:[00m Searching for tag 0x56 in stream 0x0005 (stream_type 0x06)
mpegtsbase mpegtsbase.c:291:mpegts_get_descriptor_from_stream:[00m Searching for tag 0x59 in stream 0x0005 (stream_type 0x06)
mpegtsbase mpegtsbase.c:532:mpegts_base_program_add_stream:[00m pid:0x0262, stream_type:0x006
mpegtsbase mpegtsbase.c:557:mpegts_base_program_add_stream:[00m PID 0x0262, registration_id 46424944 (FBID)
mpegtsbase mpegtsbase.c:291:mpegts_get_descriptor_from_stream:[00m Searching for tag 0x0a in stream 0x0262 (stream_type 0x06)
mpegtsbase mpegtsbase.c:291:mpegts_get_descriptor_from_stream:[00m Searching for tag 0x59 in stream 0x0262 (stream_type 0x06)
mpegtsbase mpegtsbase.c:291:mpegts_get_descriptor_from_stream:[00m Searching for tag 0x6a in stream 0x0262 (stream_type 0x06)
mpegtsbase mpegtsbase.c:291:mpegts_get_descriptor_from_stream:[00m Searching for tag 0x7a in stream 0x0262 (stream_type 0x06)
mpegtsbase mpegtsbase.c:291:mpegts_get_descriptor_from_stream:[00m Searching for tag 0x56 in stream 0x0262 (stream_type 0x06)
mpegtsbase mpegtsbase.c:291:mpegts_get_descriptor_from_stream:[00m Searching for tag 0x59 in stream 0x0262 (stream_type 0x06)
mpegtsbase mpegtsbase.c:532:mpegts_base_program_add_stream:[00m pid:0x0258, stream_type:0x015
mpegtsbase mpegtsbase.c:557:mpegts_base_program_add_stream:[00m PID 0x0258, registration_id 4b4c5641 (KLVA)
mpegtsbase mpegtsbase.c:291:mpegts_get_descriptor_from_stream:[00m Searching for tag 0x0a in stream 0x0258 (stream_type 0x15)
mpegtsbase mpegtsbase.c:291:mpegts_get_descriptor_from_stream:[00m Searching for tag 0x59 in stream 0x0258 (stream_type 0x15)
tsdemux tsdemux.c:1639:create_pad_for_stream:[00m Non-media stream (stream_type:0x15). Not creating pad
it seems that tsdemux does actually see all the streams but somehow refuses to create a pad for the klv stream.
If we only look at ffprobe's output the streams look identical, but I look at gstreamers log I notice that the klv stream has different stream types.
the working stream has stream_type=0x06 whereas the other stream has stream_type=0x15.
Checking the stream types on wikipedia tells:
ITU-T Rec. H.222 and ISO/IEC 13818-1 (MPEG-2 packetized data) privately defined (i.e., DVB subtitles/VBI and AC-3) Packetized metadataNot quite sure what I should make of this.
Unfortunately this input stream is not under our control, so we can't change it.
If we want to use a newer version of gstreamer we would have to compile it ourself.
Is there anything we can do to get the klv stream successfully?
Is this really the desired behavior of tsdemux?
EDIT 2
It seems to boil down to synchronous vs asynchronous klv as described by impleoTv.
Where the synchronous stream type is 0x15 and the asynchronous stream type is 0x06.
Unfortunately tsdemux isn't able to handle sychronous klv streams as described on their mailing list and the unmerged patch in gitlab..
So I'll probably need to have a look at either ffmpeg/libavformat or maybe another gstreamer plugin (if one exists)
As I already mentioned in the question, gstreamer's tsdemux isn't able to handle synchronous KLV data.
This leads to the following possibilites
Since my C and gstreamer knowledge is nowhere near enough to continue with the first two options I settled for the third option.