I've found some similar question on Stackoverflow but it does not really address my problem.
I'm playing multiple videos into a WebGL texture, one after another. It's based on user input, something like a web-based VJ tool.
Copying is done easily, and I have internal clock that is sync with same fps like the video I'm playing (eg. 30fps) and frames are updated correctly. Exactly like one of the answers offered in the above mentioned question. All that works well.
Texture is updated with:
gl.texSubImage2D(gl.TEXTURE_2D,0,0,0,this.gl.RGBA,this.gl.UNSIGNED_BYTE,video);
My problem is how to detect when the very first frame is available for composition. Video does not start playback immediately in the real environment (eg. average 4G, video on CDN), but sometimes takes 1s or more to start playback (to buffer sufficient data).
If I attempt to start updating texture prior the first frame is available, I get WebGL error thrown:
WebGL: INVALID_VALUE: texSubImage2D: no video
I'm aware of video.load(); method that can be called in advance (eg. on user interaction), however I have ~50 video files that I need to play (in unknown order as it depends on user input), and older phones (like iPhone 7) have major performance drop when I do that, to the point Safari sometimes crashes.
I'm looking for a reliable way to determine when video started actual playback. Events such as onplay don't seem to fire when first frame is available, but much earlier.
I have also tried ontimeupdate event, but that one does not seem to fire when first frame is available for composition, but earlier as well, just like onplay. I can see the event fired, and I start updating texture when it's fired for the first time, but at the beginning of updates it generates WebGL error (for about 0.5-1s, depending on network speed) until video actually shows the first frame. Once buffered, no errors are thrown.
This issue is more visible in 4G/mobile network. Also tested in Chrome with throttling speed. Un-throttled speed will get me 1-4 WebGL warnings prior showing first frame, while eg. 12mbps throttled will give me 100-200 warnings, prior video frame is presented.
I've seen requestVideoFrameCallback but it doesn't have coverage I need (iOS Safari not even planned anytime soon).
I'm trying to avoid updating texture if video frame is not ready for composition, but can't find a reliable way to determine when it is ready.
Any help is highly appreciated!
Alright, I have found a solution at listening on playing event.
I was listening on play and on timeupdate, didn't think of on playing being so different.
It does fire after the first frame is available for composition and now I don't have anymore those WebGL errors. Tested on an Android 10 device as well as on iOS 14.5 iPhone 7 device.