I've got a simple NDK Android code to decode and play simple video file. I have noticed that Android has native_window_set_buffers_timestamp(ANativeWindow, int64_t); method.
I read that it gives timestamps to frames added to the queue. Is there a possibility to use these stamps to cause dequeue and render frames in planned by me time moments? I've got code similar to described: in this example I'm setting Presentation Time Stamps as in example (I'm sure I've got right values, I set them in nanoseconds), but video is rendered as fast as it is decoded (way too fast). Setting timestamps doesn't have any effect. Has anyone any solution for that problem?
The field exists as a way to pass timestamps through the system. It may or may not affect how the frame is handled.
IIRC, the initial use of the field was for frames of video captured by the camera. By including the timestamp from when the frame was captured, an app that generates encoded video can output accurate timestamps. If the app used the time that the frame was received by the app, you'd get significant variation in the time values.
Older versions of Android ignore the timestamp when displaying frames. In Android 4.4 ("KitKat"), the SurfaceFlinger compositor added a feature that allowed it hold or drop frames based on the timestamp. The purpose of this was to make A/V sync easier. This is discussed at length in the comments in Grafika's "scheduled swap" activity, notably those above the doFrame() method.
In short, the goal of the feature was to allow a video player to submit each frame slightly early, so that it's ready and waiting in the composition queue when the audio playback reaches the appropriate time. Attempting to do fine-grained sync in the app itself is problematic when the system gets busy. (SurfaceFlinger runs at elevated priority.)
Providing properly-paced playback when an app spams an entire video file was not a goal. For backward compatibility with apps that feed the embedded PTS with the frames, and to avoid very long stalls if an app submits a timestamp that is years in the future, SurfaceFlinger ignores timestamps that are not within a second or so of the current time. So you still have to spoon-feed the frames, but the timestamps can help avoid stalls and drops.
So... if you want to manage your own video playback, you will have to pace the output yourself. You can see an example in Grafika's SpeedControlCallback class.