Detect current Keyframe interval in AVAsset

Question

I am working on an application that plays back video and allows the user to scrub forwards and backwards in the video. The scrubbing has to happen smoothly, so we always re-write the video with SDAVAssetExportSession with the video compression property AVVideoMaxKeyFrameIntervalKey:@1 so that each frame will be a keyframe and allow smooth reverse scrubbing. This works great and provides smooth playback. The application uses video from a variety of sources and can be recorded on android or iOS devices and even downloaded from the web and added to the application, so we end up with quite different encodings, some of which are already suited for scrubbing (each frame is a keyframe). Is there a way to detect the keyframe interval of a video file so I can avoid needless video processing? I have been through much of AVFoundation's docs and don't see an obvious way to get this information. Thanks for any help on this.

Answer 1

If you can quickly parse the file without decoding the images by creating an AVAssetReaderTrackOutput with nil outputSettings. The frame sample buffers you encounter have an attachment array containing a dictionary with useful information, include whether the frame depends on other frames, or whether other frames depend on it. I would interpret that former as indicating a keyframe, although it gives me some low number (4% keyframes in one file?). Anyway, the code:

let asset = AVAsset(url: inputUrl)
let reader = try! AVAssetReader(asset: asset)

let videoTrack = asset.tracks(withMediaType: AVMediaTypeVideo)[0]
let trackReaderOutput = AVAssetReaderTrackOutput(track: videoTrack, outputSettings: nil)

reader.add(trackReaderOutput)
reader.startReading()

var numFrames = 0
var keyFrames = 0

while true {
    if let sampleBuffer = trackReaderOutput.copyNextSampleBuffer() {
        // NB: not every sample buffer corresponds to a frame!
        if CMSampleBufferGetNumSamples(sampleBuffer) > 0 {
            numFrames += 1
            if let attachmentArray = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, false) as? NSArray {
                let attachment = attachmentArray[0] as! NSDictionary
                // print("attach on frame \(frame): \(attachment)")
                if let depends = attachment[kCMSampleAttachmentKey_DependsOnOthers] as? NSNumber {
                    if !depends.boolValue {
                        keyFrames += 1
                    }
                }
            }
        }
    } else {
        break
    }
}

print("\(keyFrames) on \(numFrames)")

N.B. This only works for local file assets.

p.s. you don't say how you're scrubbing or playing. An AVPlayerViewController and an AVPlayer?