Search code examples
androidandroid-mediaprojection

Transmit output of MediaProjection API over network


I've been reading on MediaProjection API, and checked out a few sample projects on Github that show how to use it.
All those projects only show how to save the screen capture in a file ( record the screen ), and nothing about transmiting it in real-time to another device ( screencast ).

I wasn't sure that this was possible via this API, but then I read the docs :

Screen capturing and sharing

Android 5.0 lets you add screen capturing and screen sharing capabilities to your app with the new android.media.projection APIs. This functionality is useful, for example, if you want to enable screen sharing in a video conferencing app.

The new createVirtualDisplay() method allows your app to capture the contents of the main screen (the default display) into a Surface object, which your app can then send across the network. The API only allows capturing non-secure screen content, and not system audio. To begin screen capturing, your app must first request the user’s permission by launching a screen capture dialog using an Intent obtained through the createScreenCaptureIntent() method.

For an example of how to use the new APIs, see the MediaProjectionDemo class in the sample project.

I've searched the whole day for a sample on how to use the Surface object to transmit it's data, then receive and show them on another device, but with no luck.

So the question is, how can it be done ? Can anyone point me in the right direction ?


Solution

  • Well, you can extract the frames from the surface you pass to the createVirtualDisplay() (with ImageReader which is one of the simplest ways to do that or by using SurfaceTexture, depending on your needs), after you get a bitmap, you just send it over the network (that part is application specific).

    For instance you could establish a websocket connection to your server and use it to transmit the frames (that's what I did when I played with the API), alternatively you can use WebRTC for that.