Search code examples
swiftavfoundationtvoshttp-live-streamingfairplay

Failed to get video thumbnail from AVPlayer using Fairplay HLS

I'm trying to build a custom progress bar for a video player app in tvOS, and would like to show thumbnails of the video while the user scans the video.

I'm using AVPlayer and Fairplay HLS to play remote video files. I've tried to do this using 2 methods. One with AVAssetImageGenerator's copyCGImage, and the other with AVPlayerItemVideoOutput's copyPixelBuffer method. Both return nil.

When I tried with a local video file, the first method worked.

Method 1:

let imageGenerator = AVAssetImageGenerator(asset: playerItem.asset)
let progressSeconds = playerItem.duration.seconds * Double(progress)
let time = CMTime(seconds: progressSeconds, preferredTimescale: 5)
if let imageRef = try? imageGenerator.copyCGImage(at: time, actualTime: nil) {
   image = UIImage(cgImage:imageRef)
}

Method 2:

let videoThumbnailsOutput = AVPlayerItemVideoOutput(pixelBufferAttributes: [String(kCVPixelBufferPixelFormatTypeKey): NSNumber(value: kCVPixelFormatType_32BGRA)])
player?.currentItem?.add(videoThumbnailsOutput)

if let pixelBuffer = videoThumbnailsOutput.copyPixelBuffer(forItemTime: time, itemTimeForDisplay: nil) {
   let ciImage = CIImage(cvPixelBuffer: pixelBuffer)

Any ideas what I'm doing wrong or is there any other way?

Thanks!

Solution

  • This is usually done by making use of the trick play stream associated to your actual stream.

    https://en.wikipedia.org/wiki/Trick_mode

    You can find it declared with the key EXT-X-I-FRAME-STREAM-INF in the manifest of your HLS stream. A regex might be needed in order to parse its value.

    "#EXT-X-I-FRAME-STREAM-INF[^#]*URI=[^#]*"

    Once you have the URL of the trick play stream, you can use a paused instance of AVPlayer as a thumbnail. And when the user swipe left and right, you should seek the player in the thumbnail to show the right frame.