I'm using ImageReader to get frames from phone's camera using Camera2 API.
I've set RepeatingRequest inside my onConfigured callback function and I logged image.getTimestamp inside onImageAvialable function for calculating the fps.
When I looked at the time stamps on logcat I saw that there are only 20 fps (20 time stamps per second).
My questions are:
Here are relevant parts of the code:
public void onImageAvailable(ImageReader imageReader) {
Image image = imageReader.acquireLatestImage();
Log.d(TAG, "onImageAvailable: IMAGE AVAILABLE" + image.getTimestamp() );
// If STOP CAPTURE button hasn't been pressed yet
if ((!isStopped) && (image != null)) {
// Get the current frame
ByteBuffer pixelArray1 = image.getPlanes()[0].getBuffer();
byte[] bytes = new byte[pixelArray1.remaining()];
pixelArray1.get(bytes);
Mat mat = new Mat(image.getHeight() + image.getHeight() / 2, image.getWidth(), CvType.CV_8UC1);
mat.put(0, 0, bytes);
Mat rgb = new Mat(image.getHeight(), image.getWidth(), CvType.CV_8UC4);
// OpenCV function - convert from YUV which is the camera format into RGB which can be processed
Imgproc.cvtColor(mat, rgb, Imgproc.COLOR_YUV420sp2BGR, 4);
int pixelStride = image.getPlanes()[0].getPixelStride();
int rowStride = image.getPlanes()[0].getRowStride();
int rowPadding = rowStride - pixelStride * 640;
final Bitmap bitmap = Bitmap.createBitmap(640 + rowPadding / pixelStride, 480, Bitmap.Config.RGB_565);
Utils.matToBitmap(rgb, bitmap);
runOnUiThread(new Runnable() {
@Override
public void run() {
// Update screen with new frame
imageView.setImageBitmap(bitmap);
}
});
image.close();
}
else {
closeCamera();
}
}
};
onConfigured:
public void onConfigured(CameraCaptureSession cameraCaptureSession) {
Log.d(TAG, "onConfigured: BUILD REQUEST AND CAPTURE");
try {
CaptureRequest.Builder requestBuilder = cameraCaptureSession.getDevice().createCaptureRequest(CameraDevice.TEMPLATE_RECORD);
requestBuilder.addTarget(imageReaderSurface);
//cameraCaptureSession.setRepeatingRequest(requestBuilder.build(), null, handler);
cameraCaptureSession.setRepeatingRequest(requestBuilder.build(), null, handler);
Log.d(TAG, "onConfigured: AFTER CAPTURE");
} catch (CameraAccessException e) {
Log.d(TAG, "onConfigured: CANT CREATE CAPTURE REQUEST");
e.printStackTrace();
}
}
And the logcat which shows 20 time stamps per second:
03-24 11:41:30.919 16254-17780/com.example.noamm_000.talkwithcompviawifi
D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176055988501
03-24 11:41:30.964 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176106451209
03-24 11:41:31.014 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176156344543
03-24 11:41:31.067 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176205880116
03-24 11:41:31.120 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176256464074
03-24 11:41:31.168 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176306533293
03-24 11:41:31.217 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176357056886
03-24 11:41:31.269 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176406944855
03-24 11:41:31.313 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176457030845
03-24 11:41:31.363 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176506663397
03-24 11:41:31.414 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176556684438
03-24 11:41:31.467 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176607584803
03-24 11:41:31.520 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176658480428
03-24 11:41:31.567 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176706916105
03-24 11:41:31.613 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176757571834
03-24 11:41:31.664 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176807395636
03-24 11:41:31.717 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176857408865
03-24 11:41:31.767 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176907807199
03-24 11:41:31.816 16254-17780/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onImageAvailable: IMAGE AVAILABLE39176957648657
*I tried putting in comment all of the work that being done in the onImageAvialable and leave only the time stamps logging but I still got 20 fps.
Thank you, Noam
What you are seeing is normal.
Documentation for setRepeatingRequest clearly states that "With this method, the camera device will continually capture images using the settings in the provided CaptureRequest, at the maximum rate possible." Frame rate you will get depends on many things - capture request settings, camera hardware, available cpu power, images captured etc.