Search code examples
webrtcsdprtcpeerconnection

Should I send a WebRTC answer before my side's localMedia tracks were added?

I'm building a video calling app using WebRTC which allows one peer to call another by selecting someone in the lobby. When peer A sends a call request, the other peer B can accept. At this point, WebRTC signaling starts:

  • Both peers get their local media using MediaDevices.getUserMedia()
  • Both peers create an RTCPeerConnection and attach event listeners
  • Both peers calls RTCPeerConnection.addTrack() to add their local media
  • One peer A (the impolite user) creates an offer, calls RTCPeerConnection.setLocalDescription() to set that offer as the local description, and sends it to the WebSocket server, which forwards it to the other peer B.
  • The other peer B receives this offer and adds calls RTCPeerConnection.setRemoteDescription() to record it as the remote description
  • The other peer B then creates an answer and transmits it again to the first peer A.

(Steps based on https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/Connectivity)

This flow is almost working well. In 1 out of 10 calls, I receive no video/audio from one of the peers (while both peers have working local video). In such a case, I have noticed that the answer SDP contains a=recvonly while this should be a=sendrecv under normal circumstances. I have further determined that by the time the other peer receives the offer and needs to reply with an answer, the localMedia of this side has sometimes not yet been added, because MediaDevices.getUserMedia can take a while to complete. I have also confirmed this order of operations by logging and observing that the offer sometimes arrives before local tracks were added.

I'm assuming that I shouldn't send an answer before the local media has been added?

I'm thinking of two ways to fix this, but I am not sure which option is best, if any:

  • Create the RTCPeerConnection only after MediaDevices.getUserMedia() completes. In the meantime, when receiving an offer, there is no peer connection yet, so we save offers in a buffer to process them later once the RTCPeerConnection is created.
  • When receiving an offer, and there are no localMedia tracks yet, hold off on creating the answer until the localMedia tracks have been added.

I am having difficulties to decide which solution (or another) matches best with the "Perfect Negotiation" pattern.

Thanks in advance!

Solution

  • Yes, it is good to add the stream before creating an offer if you do it 'statically', but the best way to do it is to do it in the onnegotiationneeded event because the addtrack event triggers an onnegotiationneeded event. So you should add the stream and then use the createoffer inside onnegotiationneeded. As far as the answer you can do it before with no issues, but remember that a well-established connection will let you add/remove tracks with no problems (even after the SDP has been set). You didn't post any code but remember that you also MUST exchange ice candidates. The last piece of advice, remember that all of the above IS asynchronous! so you should use promises, and await until the description is set, only THEN create an offer/answer. Hopefully, this will help