Search code examples
webglrequestanimationframe

Can WebGL fail to keep up with requestAnimationFrame?


Lets say I have a WebGL loop like this:

function draw(timestamp)
{
    // ...
    gl.drawArrays(...);

    window.requestAnimationFrame(draw);
}

I'm afraid of the possibility of the following problem:

  • My requestAnimationFrame callback "draw" gets called by the browser about 60 times per second.
  • gl.drawArrays returns immediately, but the underlying job takes the GPU longer than 1/60 of a second to execute.
  • I end up making gl.drawArrays() calls faster than the GPU can execute them.

Questions:

  1. Can this problem actually occur in theory? If so, what is the right way to deal with it? If not, then why?
  2. Is the "requestAnimationFrame callback" guaranteed to not be called until all my previous WebGL calls have completed for all the canvas elements on the page? If so, then is it documented in the specs? I couldn't find it anywhere.

Thanks in advance!


Solution

  • No, the problem can't happen.

    Should probably close this as a duplicate but to know why you should read up on the browser's event loop

    Basically the browser runs like this

       const tasks = [];
    
       // loop forever
       for (;;)  {  
         // execute all tasks
         while (tasks.length) {
           const task = tasks.shift();
           task();
         }
         goToSleepUntilThereAreNewTasks();
       }
    

    It never runs JavaScript in parallel (except for workers) so your requestAnimationFrame callback will not get called in the middle of another requestAnimationFrame callback.

    All requestAnimationFrame does is says is effectively "next time browser is going to repaint the screen add this callback to the list of tasks to run".

    All other things in the browser happen the same. When you load a page a task is added to execute your scripts. When you add a setTimeout the browser will add your callback to the list of tasks in however many milliseconds you set the timer for. When you add event listeners for things like mousemove or keydown or img.onload the browser just adds tasks to the task list when the mouse moves or a key is pressed or an image finishes loading.

    The important part is there is no parallel work from the POV of JavaScript. Each task runs to completion and the browser never processes any other task until the current task finishes running.

    As for documentation that's in the specs


    Note that the browser can not composite the page without somehow waiting for your WebGL draw calls to fill out the canvas's drawingbuffer. It can run things in parallel, not JavaScript itself but for example executing previously submitted GPU commands while JavaScript is adding new ones or doing something else, but eventually it will block.

    Consider that from the POV of JavaScript it's running requestAnimationFrames serially. One happens, it requests the next one. From the POV of the compositor, which is the code that draws all the DOM elements into the browser window, it's running in parallel to JavaScript taking the last state of the DOM and drawing the next frame in the browser. Part of that is waiting for the calls in WebGL. It doesn't need to do a hard wait like a gl.finish. It just needs to organize the GPU commands in such a way so that the compositing GPU commands happen after the last WebGL commands. The compositor itself is not going to start rendering new frames until it finished the current frame and it is the compositor that is effectively deciding it's time for a new frame, deciding it's time for a new frame is the trigger for executing a requestAnimationFrame event.

    At best the browser will double or triple buffer such that while the browser is compositing this frame it can let JavaScript start queuing up commands for the next frame so there is more parallelism but again, it's not going to issue infinite requestAnimationFrame callbacks without waiting for the frame(s) it's generating to finish one way or another.

    So no, the problem you pointed out can't happen.


    note: the section above between the 2 separators didn't exist originally. Kaiido pointed out in comment they didn't think I answered the question. I thought I did implicitly. The implications of the event loop, are that

    1. rAF events don't execute until the browser has decided it's time to draw the page
    2. It's not going to decide to draw the page again if it's currently in the middle of drawing the page for the current frame
    3. It obviously can't finish drawing the page until the commands you issued to the canvas via WebGL are finished executing.

    Step 3 seems kind of obvious and not need to be stated. How could the browser draw the page if it it didn't wait for the canvas to have the image you drew into it? From that the rest falls out from the event loop implications. Why would the browser start a new frame if it's not finished with the current one? It wouldn't. If it hasn't started a new frame then it's not going to call your rAF callback.

    I also want to be clear it's a reasonable question to ask. Like I mentioned above the browser might let JavaScript start making the next frame while it's busy compositing the current frame. If the browser does let JavaScript start the next frame in parallel and did nothing to throttle your problem could happen. One way or another the browser will synchronize with the GPU and make sure that the frames it has submitted to the GPU have finished executing before it gets too many frames ahead (1 or 2 frames). In OpenGL terms it can easily do this with sync objects. I'm not saying the browser uses sync objects. Only that the browser can do something similar to make sure it doesn't get too many frames ahead.


    I don't know if this will make things more or less confusing but Kaiido points out in a chat discussion that if you use setTimeout instead of requestAnimationFrame then in a sense you'd get the issue you are worried about.

    Let's say you do did this

    function draw(timestamp)
    {
        // ...
        gl.drawArrays(...);
    
        setTimeout(draw);
    }
    

    In this case with setTimeout of 0 (or any number under one frame) how many GPU commands get issued between composites is unknown. Maybe it calls draw 5 times between composites or 500 times. It is undefined and up to the browser.

    The thing that I'd point out is that calling draw via setTimeout many times between composites is not "frames" it's just a really obfusticated way of issuing more GPU work. There is no functional difference between draw getting called N times between composites via N setTimeouts or by just a simple for (let i = 0; i < N; ++i) { draw(); } loop. In both cases a bunch of work got added to the GPU between composites. The only difference with the setTimeout method is the browser choses N for you.

    This is actually one of the major points of using requestAnimationFrame because setTimeout has no relation to frames whatsoever.