Search code examples
iosvideoswiftuistreamros

Play back video stream via HTTP with SwiftUI

I'm trying to play back a live video stream from a local network camera, through ROS (Robot Operating System) using VideoPlayer in SwiftUI. But the stream keeps failing. Here is what I tried:

Using this article I tried the below. https://www.hackingwithswift.com/quick-start/swiftui/how-to-play-movies-with-videoplayer

Code:

VideoPlayer(player: AVPlayer(url: URL(string: "http://192.168.45.100:8080/stream?topic=/image_raw")!))

This results in black player and the console writes out:

2021-11-24 14:11:47.252729+0100 wifi-test[2965:5457477] <CATransformLayer: 0x2820ae8c0> - changing property masksToBounds in transform-only layer, will have no effect
2021-11-24 14:11:47.275318+0100 wifi-test[2965:5457477] <CATransformLayer: 0x282094c60> - changing property allowsGroupBlending in transform-only layer, will have no effect

Then I wanted to validate the video URL:
Opening the video url in VLC works great - so the url is correct.

Testing the url with AVAsset.isPlayable is returning false. So that led me to that something is wrong with the url.

let url = URL(string: "http://192.168.45.100:8080/stream?topic=/image_raw")
if AVAsset(url: url!).isPlayable {}

So I suspect it is because of the HTTP (and not HTTPS) protocol. My camera only supports HTTP, so switching to HTTPS is not an option.

I tried setting AllowArbitraryLoads in the app properties, but with no luck. enter image description here

I also tried implementing a custom player using SwiftUI baclward compatibility:

class Video {
    let realUrl = "http://192.168.45.100:8080/stream?topic=/image_raw"
    let testUrl = "https://sylvan.apple.com/Videos/comp_GL_G002_C002_PSNK_v03_SDR_PS_20180925_SDR_2K_AVC.mov"
    
    func getUrl() -> URL? {
        guard let url = URL(string: realUrl) else {
            assertionFailure("Video url not valid")
            return nil
        }
        
        guard AVAsset(url: url).isPlayable else {
            assertionFailure("Video not playable")
            return nil
        }
        
        return url
    }
    
}

struct CustomPlayer: UIViewControllerRepresentable {
    func makeUIViewController(context: Context) -> some AVPlayerViewController {
        
        let video = Video()
        let controller = AVPlayerViewController()
        let player = AVPlayer(url: video.getUrl()!)
        controller.player = player
        return controller
    }
    
    func updateUIViewController(_ uiViewController: UIViewControllerType, context: Context) {
        
    }
}

But .isPlayable still fails.

How do I make the video stream work?

Update: So apparently this url works fine in the players but fails validation AVAsset.isPLayable, so that check is nothing worth.

"http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4"

Also this working stream is HTTP, so that rules out my thery on HTTP vs. HTTPS.

It must be a compatibility issue with my stream format. Any idea how I can verify that?

Solution

  • So, I figured out that the stream is a MJPEG stream and the AVPlayer doesen't support that. Therefore it only shows the first frame.

    So I've implemented a camera service that fetches the MJPEG stream, like this:

    protocol CameraServiceDelegateProtocol {
    func frame(image: UIImage) -> Void
}

protocol CameraServiceProtocol {
    var rosServiceDelegate: CameraServiceDelegateProtocol { get set }
}

class CameraService: NSObject, ObservableObject {
    var cameraServiceDelegate: CameraServiceDelegateProtocol
    let realUrl = URL(string: "http://192.168.45.100:8080/stream?topic=/image_raw")
    var dataTask: URLSessionDataTask?
    var receivedData: NSMutableData = NSMutableData()
    var session: URLSession?
    
    init(delegate: CameraServiceDelegateProtocol) {
        cameraServiceDelegate = delegate
    }
    
    func play() {
        session = URLSession(configuration: URLSessionConfiguration.default, delegate: self, delegateQueue: nil)
        dataTask = session?.dataTask(with: realUrl!)
        dataTask?.resume()
    }
    
    func stop() {
        dataTask?.cancel()
    }
}

extension CameraService: URLSessionDataDelegate {
    func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive response: URLResponse, completionHandler: @escaping (URLSession.ResponseDisposition) -> Void) {
        if self.receivedData.length > 0,
            let receivedImage = UIImage(data: self.receivedData as Data) {
                
            DispatchQueue.main.async {
                self.cameraServiceDelegate.frame(image: receivedImage)
            }
                
            self.receivedData = NSMutableData()
        }
            
        completionHandler(URLSession.ResponseDisposition.allow) //.Cancel,If you want to stop the download
            
    }
        
    func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data) {
        self.receivedData.append(data)
    }
}