Search code examples
webrtcrtcdatachannelwebrtc-android

WebRTC Datachannel for high bandwidth application


I want to send unidirectional streaming data over a WebRTC datachannel, and is looking of the best configuration options (high BW, low latency/jitter) and others' experience with expected bitrates in this kind of application.

My test program sends chunks of 2k, with a bufferedAmountLowThreshold event callback of 2k and calls send again until bufferedAmount exceeds 16k. Using this in Chrome, I achieve ~135Mbit/s on LAN and ~20Mbit/s from/to a remote connection, that has 100Mbit/s WAN connection on both ends.

What is the limiting factor here?

How can I see if the data is truly going peer to peer directly, or whether a TURN server is used?

My ultimate application will use the google-webrtc library on Android - I'm only using JS for prototyping. Can I set options to speed up bitrate in the library, that I cannot do in official JS APIs?


Solution

  • There are many variables that impact throughput and it also highly depends on how you've measured it. But I'll list a couple of things I have adjusted to increase the throughput of WebRTC data channels.

    Disclaimer: I have not done these adjustments for libwebrtc but for my own WebRTC data channel library called RAWRTC, which btw also compiles for Android. However, both use the same SCTP library underneath, both use some OpenSSL-ish library and UDP sockets, so all of this should be appliable to libwebrtc.

    Note that WebRTC data channel implementations using usrsctp are usually CPU bound when executed on the same machine, so keep that in mind when testing. With RAWRTC's default settings, I'm able to achieve ~520 Mbit/s on my i7 5820k. From my own tests, both Chrom(e|ium) and Firefox were able to achieve ~350 Mbit/s with default settings.

    Alright, so let's dive into adjustments...

    UDP Send/Receive Buffer Size

    The default send/receive buffer of UDP sockets in Linux is quite small by default. If you can, you may want to adjust it.

    DTLS Cipher Suites

    Most Android devices have ARM processors without hardware AES support. ChaCha20 usually performs better in software and thus you may want to prefer it.

    (This is what RAWRTC negotiates by default, so I have not included it in the end results.)

    SCTP Send/Receive Buffer Size

    The default send/receive window size of usrsctp, the SCTP stack used by libwebrtc, is 256 KiB which is way too small to achieve high throughput with moderate delay. The theoretical maximum throughput is limited by mbits = (window / (rtt_ms / 1000)) / 131072. So, with the default window of window=262144 and a fairly moderate RTT of rtt_ms=20, you will end up with a theoretical maximum of 100 Mbit/s.

    The practical maximum is below that... actually, way lower than the theoretical maximum (see my test results). This may be a bug in the usrsctp stack (see sctplab/usrsctp#245).

    The buffer size has been increased in Firefox (see bug 1051685) but not in libwebrtc used by Chrom(e|ium).

    Release Builds

    Optimisation level 3 makes a difference (duh!).

    Message Size

    You probably want to send 256 KiB sized messages.

    Unless you need to support Chrome < ??? (sorry, I currently don't know where it landed...), then the maximum message size is 64 KiB (see issue 7774).

    Unless you also need to support Firefox < 56, in which case the maximum message size is 16 KiB (see bug 979417).

    It also depends on how much you send before you pause sending (i.e. the buffer's high water mark), and when you continue sending after the buffer has been drained (i.e. the buffer's low water mark). My tests have shown that targeting a high water mark of 1 MiB and setting a low water mark of 256 KiB results in adequate throughput.

    This reduces the amount of API calls and can increase throughput.

    End Results

    Using optimisation level 3 with default settings on RAWRTC brought me up to ~600 Mbit/s.

    Based on that, increasing the SCTP and UDP buffer sizes to 4 MiB brought me up further to ~700 Mbit/s, with one CPU core at 100% load.

    However, I believe there is still room for improvements but it's unlikely to be low-hanging.


    How can I see if the data is truly going peer to peer directly, or whether a TURN server is used?

    Open about:webrtc in Firefox or chrome://webrtc-internals in Chrom(e|ium) and look for the chosen ICE candidate pair. Or use Wireshark.