Edit: To help illustrate the error I keep getting I have created a CodePen of the issue I am seeing. Open up the console and you will see the error. [https://codepen.io/FifthCloud/pen/eYpqJLN]
I wanted to design my own streaming camera service in my home using React/Node. So far I have gone a long way with a Client/Server approach. The server is a raspberry pi with camera attached to it. Using FFMpeg I can see all the video from it and using websockets I can successfully send the data (I mean I see the data being sent to the client will get into why it will not render).
The issue I am running into is why cannot I append the data to the buffer. For starters, I am using this demo about Media Source as a guide and example on what I am supposed to be doing. http://nickdesaulniers.github.io/netfix/demo/bufferAll.html I got this link from my research about Media Source's here: https://developer.mozilla.org/en-US/docs/Web/API/MediaSource/readyState Also would like to mention that I have received a lot of inspiration from this post and got really far because of this answer HTML5 Video: Streaming Video with Blob URLs
With that in mind I can successfully create a MediaSource and open the buffer, but what I cannot understand is why am I unable to append to the buffer? The error I am recieving is this:
TypeError: Failed to execute 'appendBuffer' on 'SourceBuffer': No function was found that matched the signature provided.
The client side code is the following
class ProductDetail extends React.Component {
constructor(props) {
super(props);
this.handleData = this.handleData.bind(this);
this.handleOpen = this.handleOpen.bind(this);
this.appendToSourceBuffer = this.appendToSourceBuffer.bind(this);
this.mediaStreaming = new MediaSource();
this.blobArr = [];
this.buffer = null;
this.myRef = React.createRef();
console.log("buffer initialized");
console.log("ReadyState [Initialized]: " + this.mediaStreaming.readyState);
this.state = {
msg: "",
stream: "",
streaming: URL.createObjectURL(this.mediaStreaming),
};
}
componentDidMount() {
this.mediaStreaming.addEventListener("sourceopen", () => {
console.log(
"ReadyState [Source Open]: " + this.mediaStreaming.readyState
);
this.buffer = this.mediaStreaming.addSourceBuffer(
'video/mp4;codecs="avc1.42E01E"'
);
this.buffer.addEventListener("updateend", () => {
this.mediaStreaming.endOfStream();
document.getElementById("streaming").play();
console.log(
"ReadyState [UpdateEnd]: " + this.mediaStreaming.readyState
);
});
});
}
handleData(data) {
const videoStream = JSON.parse(data);
if (videoStream.msg === "videoStream") {
this.blobArr.push(videoStream.chunk.data);
if (
this.mediaStreaming.readyState === "open" &&
this.buffer &&
this.buffer.updating === false &&
this.blobArr.length > 0
) {
console.log(
"ReadyState [AppendBuffer]: " + this.mediaStreaming.readyState
);
this.buffer.appendBuffer(this.blobArr.shift()); // Error message shows at this line!
document.getElementById("streaming").play();
console.log("Buffer Updating", this.buffer.updating);
}
}
}
handleOpen() {
console.log("Status: connected");
}
handleClose() {
console.log("Status: disconnected");
}
render() {
return (
<div>
<Websocket
url="ws://192.168.1.20:5000"
onMessage={this.handleData}
onOpen={this.handleOpen}
onClose={this.handleClose}
reconnect={true}
debug={true}
ref={(Websocket) => {
this.refWebSocket = Websocket;
}}
protocol="stream-protocol"
/>
<video width="640" height="480" id="streaming" controls autoPlay>
<source
ref={this.myRef}
src={this.state.streaming}
type="video/mp4"
/>
Your browser does not support the video tag.
</video>
</div>
);
}
}
The error comes from the handleData event from my WebSocket when the server is sending the next streaming byte array data.
As I mentioned about the example and the guide from the link I have posted I looked at their code and stepped through it. According to them Append Buffer should work fine, however even in their code appendbuffer() is only listed in the proto section see here:https://drive.google.com/open?id=1vOU4oM-XoKe71DofQuLU2THxMdgiown_ and yet the code doesn't complain about their appendbuffer(). When I console log my buffer it looks like theirs https://drive.google.com/open?id=1T4Ix-y124NgQJ9xu97xv4U2yFTn2k7vF. What am I missing?
I feel like the answer is very obvious but it is escaping me as to what I am doing wrong.
I'm building a similar project myself, using several high-end RTSP-enabled IP cameras, and just ran into the same error today. First of all, keep in mind that this API is experimental, according to the Mozilla Developer Network (MDN) documentation (link below).
That being said, I seem to have found the root cause of the issue, even though I'm not a JavaScript expert by any stretch. It turns out that you can't directly pass the "Blob" object from data.data in handleData() into the appendBuffer() method. Instead, you need to convert the Blob to a data type that's supported by the appendBuffer() method first, before passing it in.
async functionBlob.arrayBuffer() to convert the data blobs into a JavaScript ArrayBufferChange this:
handleData(data) {
const videoStream = JSON.parse(data);
if (videoStream.msg === "videoStream") {
this.blobArr.push(videoStream.chunk.data);
if (
this.mediaStreaming.readyState === "open" &&
this.buffer &&
this.buffer.updating === false &&
this.blobArr.length > 0
) {
console.log(
"ReadyState [AppendBuffer]: " + this.mediaStreaming.readyState
);
this.buffer.appendBuffer(this.blobArr.shift()); // Error message shows at this line!
document.getElementById("streaming").play();
console.log("Buffer Updating", this.buffer.updating);
}
}
}
Into this:
async handleData(event) {
var buffer = await event.data.arrayBuffer(); // convert the message Blob into an ArrayBuffer
appendBuffer(buffer);
document.getElementById("streaming").play();
console.log("Buffer Updating", this.buffer.updating);
}
The problem is, once you do this, you'll start getting a new error, because there is too much data being passed into the SourceBuffer object on the MediaSource. Since it can't keep up with the input stream, you'll see something similar to the following error:
Failed to execute 'appendBuffer' on 'SourceBuffer': This SourceBuffer is still processing an 'appendBuffer' or 'remove' operation
It seems like we need to "buffer the buffers," as odd as that sounds.
I haven't solved this new error yet. If you find a solution, I would love some help.