Andriod Studio OpenCV 3.2, Cannot open video file or android camera with native OpenCV

Question

I'm currently working on Android native image processing with camera/video.

I'm using OpenCV 3.2 to capture and process image.
The problem is that I cannot open any kind of video sources or camera!

This is my c++ code.

cv::VideoCapture cap;
cv::Mat frame;

JNIEXPORT void JNICALL
Java_com_project_vmtest_Native_openVideo(JNIEnv *env, jobject instance, jstring fileName) {

    const char * fileNameNative;
    jboolean isCopy;
    fileNameNative = env->GetStringUTFChars(fileName, &isCopy);

    cap = cv::VideoCapture(fileNameNative);
    cap.open(fileNameNative);
    if(cap.isOpened())        // ALWAYS FALSE HERE!!!!
        cap.read(frame);
}

JNIEXPORT void JNICALL
Java_com_project_vmtest_Native_render(JNIEnv *env, jclass type) {
    if(engine){
        cap.read(frame);
        if (frame.empty())      return;    // ALWAYS RETURNS HERE!!!!
        engine->prepare(frame);
        engine->renderObjToFBO();
        engine->renderFrame();
        engine->clearFrameData();
    }
}

I checked fileName and it seems to be proper.
i.e. This is my file structure:

app\src\main\cpp\native_main.cpp    // native code
app\src\main\assets\vmData\test\testFile.mp4   // video file

Then the fileName is:

fileName = "../assets/vmData/test/testFile.mp4"

I tried with "..\assets\vmData\test\testFile.mp4" and nothing changed.

I tried cap.open(CV_CAP_ANDROID + 0); to grab camera preview and got the same result, VideoCapture won't open at all!

I enabled the Android camera permission with below code.

<uses-permission android:name="android.permission.CAMERA"/>
<uses-feature android:name="android.hardware.camera"/>
<uses-feature android:name="android.hardware.camera.autofocus"/>
<uses-feature android:name="android.hardware.camera.front"/>
<uses-feature android:name="android.hardware.camera.front.autofocus"/>

My test device is Android 5.0 Lollypop so I don't think it's any kind of permission issue.

Struggling for hours.
Any helps are appreciated!

Answer 1

I failed to open camera inside Native code, instead I ended up using android OpenCV Camera implementation, opencv.android.CameraBridgeViewBase.

MainActivity.java

public class MainActivity extends AppCompatActivity implements CameraBridgeViewBase.CvCameraViewListener2 {

    private CameraBridgeViewBase mCvCamView;
    Mat matInput, matOutput;

    private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
        @Override
        public void onManagerConnected(int status) {
            switch (status) {
                case LoaderCallbackInterface.SUCCESS:
                    mCvCamView.enableView();
                    break;
                default:
                    super.onManagerConnected(status);
                    break;
            }
        }
    };


    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        mCvCamView = (CameraBridgeViewBase)findViewById(R.id.cam_view);
        mCvCamView.setVisibility(SurfaceView.VISIBLE);
        mCvCamView.setCvCameraViewListener(this);
        mCvCamView.setCameraIndex(1); // front-camera(1),  back-camera(0)
        mLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);
    }

    @Override
    public void onPause()
    {
        super.onPause();
        if (mCvCamView != null)
            mCvCamView.disableView();
    }

    @Override
    public void onResume()
    {
        super.onResume();

        if (!OpenCVLoader.initDebug()) {
            Log.d(TAG, "onResume :: Internal OpenCV library not found.");
            OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_2_0, this, mLoaderCallback);
        } else {
            Log.d(TAG, "onResume :: OpenCV library found inside package. Using it!");
            mLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);
        }
    }

    public void onDestroy() {
        super.onDestroy();
        if (mCvCamView != null)
            mCvCamView.disableView();
    }

    @Override
    public void onCameraViewStarted(int width, int height) {
        if ( matOutput == null || matOutput.empty() )
            matOutput = new Mat(height, width, CvType.CV_8UC4, new Scalar(255, 0, 0, 255));
    }

    @Override
    public void onCameraViewStopped() {}

    @Override
    public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
        matInput = inputFrame.rgba();
        Native.extract(matInput.getNativeObjAddr(), matOutput.getNativeObjAddr());
        return matOutput;
    }

}

activity_main.xml

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent" >

    <org.opencv.android.JavaCameraView
        android:id="@+id/cam_view"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

</RelativeLayout>