Stream video with bitmap as overlay

Question

I'm new to wowza and is working on a project to live stream video captured from an Android device. I need to attach an image(dynamic one) to the video stream so that the users watching the stream can view it. The code I have tried is given below(as from the example source code from wowza):

        // Read in a PNG file from the app resources as a bitmap
        Bitmap overlayBitmap = BitmapFactory.decodeResource(getResources(), R.drawable.overlay_logo);

        // Initialize a bitmap renderer with the bitmap
        mWZBitmap = new WZBitmap(overlayBitmap);

        // Place the bitmap at top left of the display
        mWZBitmap.setPosition(WZBitmap.LEFT, WZBitmap.TOP);

        // Scale the bitmap initially to 75% of the display surface width
        mWZBitmap.setScale(0.75f, WZBitmap.SURFACE_WIDTH);

        // Register the bitmap renderer with the GoCoder camera preview view as a frame listener
        mWZCameraView.registerFrameRenderer(mWZBitmap);

This works fine, but I don't want to show the image at the broadcasting end, the image should be visible only at the receiving end. Is there anyway to get this done?

Answer 1

I managed to get this done by registeringFrameRenderer and setting the bitmap inside onWZVideoFrameRendererDraw.

Code snippet is as given below(Kotlin):

private fun attachImageToBroadcast(scoreValue: ScoreUpdate) {
    bitmap = getBitMap(scoreValue)
    // Initialize a bitmap renderer with the bitmap
    mWZBitmap = WZBitmap(bitmap)
    // Position the bitmap in the display
    mWZBitmap!!.setPosition(WZBitmap.LEFT, WZBitmap.TOP)
    // Scale the bitmap initially
    mWZBitmap!!.setScale(0.37f, WZBitmap.FRAME_WIDTH)
    mWZBitmap!!.isVisible = false // as i dont want to show it initially
    mWZCameraView!!.registerFrameRenderer(mWZBitmap)
    mWZCameraView!!.registerFrameRenderer(VideoFrameRenderer())
}

private inner class VideoFrameRenderer : WZRenderAPI.VideoFrameRenderer {
    override fun onWZVideoFrameRendererRelease(p0: WZGLES.EglEnv?) {
    }

    override fun onWZVideoFrameRendererDraw(p0: WZGLES.EglEnv?, framSize: WZSize?, p2: Int) {
            mWZBitmap!!.setBitmap(bitmap) // note that the bitmap value gets changed once I get the new values
                                          //I have implemented some flags and conditions to check whether a new value has been obtained and only if these values are satisfied, the setBitmap is called. Otherwise, as it is called continuously, flickering can occur in the screen
            }


    override fun isWZVideoFrameRendererActive(): Boolean {
        return true
    }

    override fun onWZVideoFrameRendererInit(p0: WZGLES.EglEnv?) {
    }

}

In iOS, we can implement WZVideoSink protocol to achieve this. First, we need to update the scoreView with the latest score and then convert the view to an image. Then we can embed this image to the captured frame using WZVideoSink protocol method. A sample code is given below.

 // MARK: - WZVideoSink Protocol
func videoFrameWasCaptured(_ imageBuffer: CVImageBuffer, framePresentationTime: CMTime, frameDuration: CMTime) {

        if self.goCoder != nil && self.goCoder!.isStreaming {
           let frameImage = CIImage(cvImageBuffer: imageBuffer)
            var addCIImage: CIImage = CIImage()

            if let scoreImage = self.getViewAsImage() {
 // scoreImage is the image you want to embed.
                addCIImage = CIImage(cgImage: scoreImage.cgImage!)
            }

            let filter = CIFilter(name: "CISourceOverCompositing")
            filter?.setDefaults()
            filter?.setValue(addCIImage, forKey: kCIInputImageKey)
            filter?.setValue(frameImage, forKey: kCIInputBackgroundImageKey)

            if let outputImage: CIImage = filter?.value(forKey: kCIOutputImageKey) as? CIImage {
                let context = CIContext(options: nil)
                context.render(outputImage, to: imageBuffer)
            } else {
                let context = CIContext(options: nil)
                context.render(frameImage, to: imageBuffer)
            }
        }
}

func getViewAsImage() -> UIImage {
// convert scoreView to image
    UIGraphicsBeginImageContextWithOptions(self.scoreView.bounds.size, false, 0.0)
    self.scoreView.layer.render(in: UIGraphicsGetCurrentContext()!)

    let scoreImage: UIImage = UIGraphicsGetImageFromCurrentImageContext()!
    UIGraphicsEndImageContext()
    return scoreImage
}