Search code examples
javascriptwebrtcfile-sharingchunkspeerjs

PeerJS/WebRTC connection fails on rapid chunks transmittion


I'm using PeerJS, but thought that this problem can be about WebRTC in general, hope You can help me out:

I'm trying to write a simple peer-to-peer file sharing. I'm using serialisation: "none" for PeerJS connection DataChannel, as I'm sending just pure ArrayBuffers.
Everything is good with files around 10mb but I have problems sending bigger file (30+ mb), for example after sending aroung 10-20 first chunks of 900mb zip file connection between peers start throwing Connection is not open. You should listen for the "open" event before sending messages. (on the Sender side)

My setup:

File dragged to drag&drop, Sender uses FileReader to read it as ArrayBuffer in chunks of 64x1024 bytes (no difference with 16x1024) and as soon as each chunk is read - it's sent via peer.send(ChunkArrayBuffer).

Reciever creates blob from each recieved chunk, after transmission finished creates a complete blob out of those and gives a link to user.

My peer connection settings:

   var con = peer.connect(peerid, {
        label: "file",
        reliable: true,
       serialization: "none"
   })

My sending function:

function sliceandsend(file, sendfunction) {
    var fileSize = file.size;
    var name = file.name;
    var mime = file.type;
    var chunkSize = 64 * 1024; // bytes
    var offset = 0;

 function readchunk() {
    var r = new FileReader();
    var blob = file.slice(offset, chunkSize + offset);
    r.onload = function(evt) {
        if (!evt.target.error) {
            offset += chunkSize;
            console.log("sending: " + (offset / fileSize) * 100 + "%");
            if (offset >= fileSize) {
                con.send(evt.target.result); ///final chunk
                console.log("Done reading file " + name + " " + mime);
                return;
            }
            else {                    
                con.send(evt.target.result);
            }               
        } else {
            console.log("Read error: " + evt.target.error);
            return; 
        }
        readchunk();
       };
        r.readAsArrayBuffer(blob);
    }
    readchunk();
  }

Any ideas what can cause this?

Update: Setting 50ms Timeout between chunk transmittions helped a bit, 900mb file loading reached 6% (instead of 1 - 2% previously) before started throwing errors. Maybe it's some kind of limit of simultaneous operations through datachannel or overflowing some kind of datachannel buffer?
Update1: Here's my PeerJS connection object with DataChannel object inside it:
Object visualization in Google Chrome


Solution

  • Good News everyone!
    It was a buffer overflow of DataChannel problem, thx to this article http://viblast.com/blog/2015/2/25/webrtc-bufferedamount/

    bufferedAmount is a property of DataChannel(DC) object which in the latest Chrome version displays amount of data in bytes being currently in buffer, when it exceedes 16MB - DC is silently closed. Therefore anyone who will encounter this problem need to implement buffering mechanism on application level, which will watch for this property and hold back messages if needed. Also, be aware that in versions of Chrome prior to 37 the same property displays quantity(not size) of messages, and more of that it's broken under windows and displays 0, but with v<37 on overflow DC is not closed - only exception thrown, which can also be caught to indicate buffer overflow.

    I made an edit in peer.js unminified code for myself, here you can see both methods in one function (for more of the source code you can look at https://github.com/peers/peerjs/blob/master/dist/peer.js#L217)

    DataConnection.prototype._trySend = function(msg) {
    var self = this;
    function buffering() {
        self._buffering = true;
        setTimeout(function() {
            // Try again.
            self._buffering = false;
            self._tryBuffer();
        }, 100);
        return false;
    }
    if (self._dc.bufferedAmount > 15728640) {
        return buffering(); ///custom buffering if > 15MB is buffered in DC 
    } else {
        try {
            this._dc.send(msg);
        } catch (e) {
            return buffering(); ///custom buffering if DC exception caught
        }
        return true;
     }        
    }
    

    Also opened an issue on PeerJS GitHub: https://github.com/peers/peerjs/issues/291