How to get a pointer to each frame data during preview into Android Camera2 API?

Question

I'm trying to get the video frame data frame during an preview using camera2 API. How do I get to pointer to each frame data. Before displaying the frame preview on this site, send the frame data pointer together with a structure containing the video data format to a C ++ function (JNI native). It's possible? Please help me by giving me some suggestions? I thanks in advance.

Answer 1

Regarding to get frames from preview in Java, you can add an ImageReader to your captures. Especially, you will need to impolement ImageReader.OnImageAvailableListener (doc here). From that ImageReader you can get raw bytes using code similar to that:

ImageReader.OnImageAvailableListener mOnImageAvailableListener =
        new ImageReader.OnImageAvailableListener() {

            @Override
            public void onImageAvailable(ImageReader reader) {
                Image img = null;
                img = reader.acquireLatestImage();
                Result rawResult = null;
                try {
                    if (img == null) throw new NullPointerException("cannot be null");
                    ByteBuffer buffer = img.getPlanes()[0].getBuffer();
                    byte[] data = new byte[buffer.remaining()];
                    buffer.get(data);

                    Bitmap b = BitmapFactory.decodeByteArray(byteArray, 0, byteArray.length);

                } catch (ReaderException ignored) {                        
                } catch (NullPointerException ex) {
                } finally {
                    mQrReader.reset();
                    if (img != null)
                        img.close();
                }
            }
};

You must create an ImageReader, and set its listener

mImageReader = ImageReader.newInstance(...);
mImageReader.setOnImageAvailableListener(mOnImageAvailableListener, (...));

And you can add to your capture session in this way:

Surface surface = new Surface(texture);
Surface mImageSurface = mImageReader.getSurface();
mPreviewRequestBuilder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
mPreviewRequestBuilder.addTarget(mImageSurface);
mPreviewRequestBuilder.addTarget(surface);

Please note that this is pseudocode to guide you, probably it's not ready to paste and work directly, but will provide you a good guide on how to proceed.