JavaCV read video and record

Question

I am trying to merge mp4 and mp3 on Android, I am using the JavaCV, please check my code below first, then I explained the strange error after:

private void test2() throws Exception {
    String path = Environment.getExternalStorageDirectory()
            .getAbsolutePath();

    int testId = 4;
    String videoPath = path + "/" + "sample" + testId + ".mp4";
    String audioPath = path + "/" + "love.mp3";
    String outputPath = path + "/" + "out" + testId + ".mp4";

    FrameGrabber grabber1 = new FFmpegFrameGrabber(videoPath);
    FrameGrabber grabber2 = new FFmpegFrameGrabber(audioPath);

    grabber1.start();
    grabber2.start();

    FrameRecorder recorder = new FFmpegFrameRecorder(outputPath,
            grabber1.getImageWidth(), grabber1.getImageHeight(),
            grabber2.getAudioChannels());

    double frameRate = grabber1.getFrameRate();
    recorder.setFrameRate(frameRate);
    recorder.setSampleRate(grabber2.getSampleRate());
    recorder.setVideoQuality(1);

    recorder.start();

    Frame frame1, frame2 = null;

    // getLengthInTime is correct, but getLengthInFrames not accurate.
    Log.d(TAG, " Video lengthInTime:" + grabber1.getLengthInTime()
            + " Video frames:" + grabber1.getLengthInFrames());

    // Record video.
    int count = 0;
    while (true) {
        frame1 = grabber1.grabFrame();
        if (frame1 == null) {
            break;
        }

        frame1.samples = null;
        recorder.record(frame1);
        count++;

        Log.d(TAG, "Video frame timestamp:" + grabber1.getTimestamp());
    }
    Log.d(TAG, " Video frame count:" + count);

    // Record audtio.
    long videoTimestamp = recorder.getTimestamp();
    while (true) {
        frame2 = grabber2.grabFrame();
        if (frame2 != null && grabber2.getTimestamp() <= videoTimestamp) {
            frame2.image = null;
            recorder.record(frame2);

            // Log.d(TAG, "Audio frame timestamp:" +
            // grabber2.getTimestamp());
        } else {
            break;
        }
    }

    // release
    recorder.stop();
    recorder.release();

    grabber1.stop();
    grabber2.stop();
}

The output's audio is OK, but the video is strangle. The video play 1s and stop 1s, then repeat like this. The input video I recorded by Phone's camera.

I tried to count the real number of video frames, and I found the real number is much bigger than the number got from method getLengthInFrames().

Answer 1

frameGrabber grabs both ImageFrames and Sound Frames

if you need to work on VideoFrames or its count, Do like this

Frame grabFrame = frameGrabber.grabFrame();
if (grabFrame == null) {
                        // all frames are processed
                        System.out.println("!!! Failed cvQueryFrame");
                        runOnUiThread(new Runnable() {

                            @Override
                            public void run() {
                                Toast.makeText(RecordActivity.this, "Done !!!", Toast.LENGTH_SHORT).show();
                            }

                        });
                        break;
                    }
                    if (grabFrame.image != null) {
                           //This is a video Frame, Do your work here 
 }

}