AVPlayer freezes when seeking between two videos of AVComposition

Question

Me and my team have an issue that we can't fix for few weeks now.

When we seek forward between two videos in AVComposition - the preview freezes (AVPlayer with AVPlayerLayer used for playback). It gets stuck on last frame of first video. It doesn’t freeze if it’s a simple playback (not a seek) nor if seek is quick.

Here is screen recording of what's happening: https://www.dropbox.com/s/hxk33u3id7eug2i/AVCompositionFreezeScreenRecording.mov?dl=0

It feels like we tried everything, and nothing helps. When adding second video, we ask AVMutableComposition for a compatible track, and it returns us existing track, thus we conclude that both assetTrack's are compatible.

All the ranges and duration checked many times.

It fails both when videoComposition is set on playerItem and when not.

My current theory is even tho composition says that existing compositionTrack is compatible for second video, we can’t just put second video in it for some reason, maybe the transform is incompatible or I don’t know. One more note - if we take range of source videoAssetTrack with duration that is shorter than videoAssetTrack.timeRange.duration - then everything works. Maybe some issue with segment time mapping, but anything that we tried with it failed.

I tried to minimize the amount of code needed to demonstrate the issue, so hopefully it will be easy to see what I’m talking about. Just seek slowly from end of video1 to start of video2 and it will get stuck. https://www.dropbox.com/s/c6guzplt5mq4zpi/AVCompositionFreezeDemo.zip?dl=0

Thanks very much in advance, any help would be much appreciated!

Answer 1

I found how to solve this problem with a trick when creating an AVComposition. Namely, when creating AVComposition, I began to use two instead of one AVCompositionTrack in a checkerboard pattern. This solves the problem of not only different fps for neighboring videos, but also a different transform, if it was applied.

let videoTracks = [
    composition.addMutableTrack(withMediaType: .video, preferredTrackID: kCMPersistentTrackID_Invalid)!, 
    composition.addMutableTrack(withMediaType: .video, preferredTrackID: kCMPersistentTrackID_Invalid)!
]

let timeRangeStart: CMTime = .zero
for (index, asset) in assets.enumerated() {
    let videoTrack = videoTracks[index % 2]
    let assetTrack = asset.tracks(withMediaType: .video).first
    let timeRange = CMTimeRange(start: .zero, duration: asset.duration)
    do {
        try videoTrack.insertTimeRange(
            timeRange,
            of: assetTrack,
            at: timeRangeStart
         )
         timeRangeStart = timeRangeStart + timeRange.duration
    } catch {
        logError()
    }
}