Search code examples
swiftmetal

Drawable presented late, causes steady state delay

I have a little Swift playground that uses a Metal compute kernel to draw into a texture each time the mouse moves. The compute kernel runs very fast, but for some reason, as I start dragging the mouse, some unknown delays build up in the system and eventually the result of each mouse move event is displayed as much as 4 frames after the event is received.

All my code is here: https://github.com/jtbandes/metalbrot-playground

I copied this code into a sample app and added some os_signposts around the mouse event handler so I could analyze it in Instruments. What I see is that the first mouse drag event completes its compute work quickly, but the "surface queued" event doesn't happen until more than a frame later. Then once the surface is queued, it doesn't actually get displayed at the next vsync, but the one after that.

The second mouse drag event's surface gets queued immediately after the compute finishes, but it's now stuck waiting for another vsync because the previous frame was late. After a few frames, the delay builds and later frames have to wait a long time for a drawable to be available before they can do any work. In the steady state, I see about 4 frames of delay between the event handler and when the drawable is finally presented.

  1. What causes these initial delays and can I do something to reduce them?
  2. Is there an easy way to prevent the delays from compounding, for example by telling the system to automatically drop frames?
Solution

  • I still don't know where the initial delay came from, but I found a solution to prevent the delays from compounding.

    It turns out I was making an incorrect assumption about mouse events. Mouse events can be delivered more frequently than the screen updates — in my testing, often there is less than 8ms between mouse drag events and sometimes even less than 3ms, while the screen updates at ~16.67ms intervals. So the idea of rendering the scene on each mouse drag event is fundamentally flawed.

    A simple way to work around this is to keep track of queued draws and simply don't begin drawing again if another drawable is still queued. For example, something like:

    var queuedDraws = 0

// Mouse event handler:
if queuedDraws > 1 {
  return // skip this frame
}
queuedDraws += 1

// While drawing
drawable.addPresentedHandler { _ in
  queuedDraws -= 1
}