MediaCodec's Persistent Input Surface unsupported by Camera2 Session?

Question

I am writing an Android app that supports saving RAW/JPEG and records videos at the same time. I tried passing 4 surfaces when creating CameraCaptureSession: preview, 2x ImageSaver and 1x PersistentInputSurface created by MediaCodec#createPersistentInputSurface. By using persistent input surface, I intend to avoid a stoppage between 2 captures.

When creating the session it fails with:

W/CameraDevice-JV-0: Stream configuration failed due to: endConfigure:380: Camera 0: Unsupported set of inputs/outputs provided
Session 0: Failed to create capture session; configuration failed

I have tried taking out all other surfaces, leaving only the PersistentInputSurface, still fails.

 @Override
    public void onResume() {
        super.onResume();

        //Some other setups...

        if (persistentRecorderSurface == null) {
            persistentRecorderSurface = MediaCodec.createPersistentInputSurface();
        }

        startBackgroundThread();
        startCamera();

        if (mPreviewView.isAvailable()) {
            configureTransform(mPreviewView.getWidth(), mPreviewView.getHeight());
        } else {
            mPreviewView.setSurfaceTextureListener(mSurfaceTextureListener);
        }
        if (mOrientationListener != null && mOrientationListener.canDetectOrientation()) {
            mOrientationListener.enable();
        }
    }

    private void createCameraPreviewSessionLocked() {
        try {

            SurfaceTexture texture = mPreviewView.getSurfaceTexture();
            texture.setDefaultBufferSize(mPreviewSize.getWidth(), mPreviewSize.getHeight());
            Surface surface = new Surface(texture);

            mPreviewRequestBuilder = mBackCameraDevice.createCaptureRequest(
                    CameraDevice.TEMPLATE_PREVIEW);
            mPreviewRequestBuilder.addTarget(surface);

            mBackCameraDevice.createCaptureSession(Arrays.asList(
                                surface,
                                mJpegImageReader.get().getSurface(),
                                mRAWImageReader.get().getSurface(),
                                persistentRecorderSurface
                        ), new CameraCaptureSession.StateCallback() {

                    @Override
                    public void onConfigured(CameraCaptureSession session) {
                        synchronized (mCameraStateLock) {
                            if (mBackCameraDevice == null) {
                                return;
                            }

                            try {
                                setup3AControlsLocked(mPreviewRequestBuilder);

                                session.setRepeatingRequest(mPreviewRequestBuilder.build(),
                                        mPreCaptureCallback, mBackgroundHandler);
                                mState = CameraStates.PREVIEW;
                            } catch (CameraAccessException | IllegalStateException e) {
                                e.printStackTrace();
                                return;
                            }

                            mSession = session;
                        }
                    }

                    @Override
                    public void onConfigureFailed(CameraCaptureSession session) {
                        showToast("Failed to configure camera.");
                    }
                }, mBackgroundHandler);

        } catch (CameraAccessException e) {
            e.printStackTrace();
        }
    }

Answer 1

It'd be helpful to see the system log lines right before that error line to confirm, but most likely:

You need to actually tie the persistentRecorderSurface to a MediaRecorder or MediaCodec, and call prepare() on those, before you create the camera capture session.

Otherwise, there's nothing actually at the other end of the persistent surface, and the camera can't tell what resolution or other settings are required.

Also keep in mind that there are limits on how many concurrent outputs you can have from the camera, depending on its supported hardware level and capabilities. There is currently no requirement that a device must support your combination of outputs (preview, record, JPEG, RAW), unfortunately, so it's very likely many or all devices will still give you an error.