Search code examples
androidandroid-mediarecordercircular-bufferscreencastandroid-mediaprojection

What would be the most efficient way to buffer the last X minutes of MediaProjection


I'm having a bit of trouble thinking of an efficient solution. There are a few problems I am foreseeing, the first being...

OOM Prevention

If I wanted the past 30 seconds or even 5 minutes it's doable, but what if I wanted the past 30 minutes or full hour, or maybe EVERYTHING? Keeping a byte buffer means storing it in RAM. Storing over a hundred megabytes sounds like Virtual Memory suicide.

Okay so what if we store a Y amount of time, say 30 seconds, of the previously recorded media to disk in some tmp file. That potentially could work and I can use a library like mp4 parser to concatenate them all when finished. However...

If we have 30 minutes worth that's about 60 30-second clips. This seems like a great way to burn through an SD card and even if that's not a problem I can't imagine the time needed to concatenate over a hundred files into one.

From what I've been researching, I was thinking of using local sockets to do something like...

MediaRecorder -> setOutputFile(LocalSocket.getFD())

Then in the local socket...

LocalSocket -> FileOutputStream -> write(data, position, bufsiz) -> flush()

Where the background thread handles writing and keeping track of the position, and the buffer.

This is purely pseudocode and I'm not far enough in yet to test this, am I going in the right direction with this? From what I'm thinking, this only keeps one file which gets overwritten. As it only gets written to once every Y seconds it minimized IO overhead and also minimizes the amount of RAM it eats up.

Video Length to Buffer Size

How would I obtain the size the buffer should be from requested video size. It's strange since I see some long videos that are small but short videos that are huge. So I don't know how to accurately determine this. Anyone know how I can predict this if I know the video length, encoding, etc which gets set up from Media Recorder?

Examples

Does anyone know of any examples of this? I don't think the idea is entirely original but I don't see a lot of them out there and if it does it is closed source. An example goes a long way.

Thanks in advance


Solution

  • The "continuous capture" Activity in Grafika does this, using MediaCodec rather than MediaRecorder. It keeps the last N seconds of video in a circular buffer in memory, and writes it to disk when requested by the user.

    The CircularEncoder constructor estimates the memory requirements based on target bit rate. At a reasonable rate (say 4Mbps) you'd need 1.8GB to store an hour worth of video, so that's not going to fit in RAM on current devices. Five minutes is 150MB, which is pushing the bounds of good manners. Spooling out to a file on disk is probably necessary.

    Passing data through a socket doesn't buy you anything that you don't get from an appropriate java.util.concurrent data structure. You're just involving the OS in a data copy.

    One approach would be to create a memory-mapped file, and just treat it the same way CircularEncoder treats its circular buffer. In Grafika, the frame data goes into a single large byte buffer, and the meta-data (which tells you things like where each packet starts and ends) sits in a parallel array. You could store the frame data on disk, and keep the meta-data in memory. Memory mapping would work for the five-minute case, but generally not for the full hour case, as getting a contiguous virtual address range that large can be problematic.

    Without memory-mapped I/O the approach is essentially the same, but you have to seek/read/write with file I/O calls. Again, keep the frame metadata in memory.

    An additional buffer stage might be necessary if the disk I/O stalls. When writing video data through MediaMuxer I've seen periodic one-second stalls, which is more buffering than MediaCodec has, leading to dropped frames. You can defer solving that until you're sure you actually have a problem though.

    There are some additional details you need to consider, like dropping frames at the start to ensure your video starts on a sync frame, but you can see how Grafika solved those.