I've been trying to have a live RTSP streaming of video and audio over a single stream.
What I did was very simillar to their example (creation of ServerMediaSession and adding two SubSessions to it, one for video and one for audio), the only change I've made is that I've made a new bytestream source called TcpSource, which is very simillar to their ByteStreamFileSource, only instead of "fread()", I call recv() in doReadFromFile method (video and audio have different sockets, off course).
The result is that if I have each subsession seperately, it works fine. However, when I try to stream both video and audio via TCP, it causes either major packet losses, or having only one stream working properly while the other stuck in the middle (e.g video is freezing while audio keeps playing fine).
Can you please advise? Does it has something to do with sending\receiving timeouts? Thanks in advance.
The live555 event loop is single-threaded and is sensitive to any blocking or heavy processing.
The first thing to check is that you are not using blocking I/O. If you are, you are interfering with all the things the live555 framework takes care of, e.g. any timers, RTCP reporting, socket reads and writes, etc.
The second thing is that you need to check that you are not blocking in your doReadFromFile. System calls like sleep are not recommended. Instead use the live555 task scheduler mechanism.
I have written similar code i.e. reading audio/video from tcp sockets, parsing the packets, and then re-broadcasting the media via live555. I handled all my network I/O in a separate thread and passed parsed media samples into the live555 thread via a shared queue mechanism. This has worked well so far, but there are lots of other ways you could approach this e.g. using the live555 thread for I/O like you are doing should be fine as long as you don't interfere with the live555 event loop.