For context: I'm working on an app that evaluates video to audio sync, so it's very timing sensitive.
Recording, decoding and evaluting works fine, and accurately, on my Pixel 2XL, but problems on other phones include:
In firebase testing, it causes native crashes on two separate phones (OnePlus 3t and Razer 2 - not on OnePlus 5t or above - API 28).
On a Samsung SM-G975 (API 28) it causes the display to freeze, but records video.
On a Huiwei 3P it returns extremely inaccurate results (8 or 9 frames out).
I'm opening the CameraCapture session likewise:
private void setupHighSpeedSession() {
try {
openCloseLock.acquire();
mCaptureSession.abortCaptures();
mImageReader.close();
mCaptureRequestBuilder = null;
mCaptureRequestBuilder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_RECORD);
List<Surface> surfaceList = new ArrayList<>();
Surface recorderSurface = mRecordAnalyze.getRecorder().getSurface();
surfaceList.add(recorderSurface);
mCaptureRequestBuilder.addTarget(recorderSurface);
SurfaceTexture previewTexture = preview.getSurfaceTexture();
previewTexture.setDefaultBufferSize(highSpeedSize.getWidth(), highSpeedSize.getHeight());
Surface previewSurface = new Surface(previewTexture);
surfaceList.add(previewSurface);
mCaptureRequestBuilder.addTarget(previewSurface);
mCaptureRequestBuilder.set(CaptureRequest.CONTROL_AE_TARGET_FPS_RANGE,
highSpeedFrameRate);
mCaptureRequestBuilder.setTag(CAMERA_HIGHSPEED_TAG);
if (!openCloseLock.tryAcquire(5000, TimeUnit.MILLISECONDS)) {
throw new RuntimeException("Timeout waiting for mCamera lock");
}
mCameraDevice.createConstrainedHighSpeedCaptureSession(surfaceList, mCaptureSessionListener, mBackgroundHandler);
} catch (CameraAccessException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
openCloseLock.release();
}
And setting up MediaRecorder:
void setupRecorder() throws IOException {
mRecorder = new MediaRecorder();
mRecorder.setAudioSource(mAudioProcessor.getAudioInput());
mRecorder.setVideoSource(MediaRecorder.VideoSource.SURFACE);
mRecorder.setOutputFormat(MediaRecorder.OutputFormat.DEFAULT);
mRecorder.setVideoFrameRate(mCamera.getHighSpeedFrameRate());
mRecorder.setVideoEncoder(MediaRecorder.VideoEncoder.DEFAULT);
mRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.DEFAULT);
mRecorder.setAudioChannels(1);
Size highSpeedSize = mCamera.getHighSpeedSize();
Log.i(TAG, "RecAnalyze Size: " + highSpeedSize);
mRecorder.setVideoSize(highSpeedSize.getWidth(), highSpeedSize.getHeight());
recordFile = getRecorderPath();
Log.i(TAG, "RecAnalyze Path: " + recordFile.getAbsolutePath());
mRecorder.setOutputFile(recordFile.getAbsolutePath());
Polling camera for highspeed capabilities:
if (cameraCharacteristics.get(CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES)[i] ==
CameraMetadata.REQUEST_AVAILABLE_CAPABILITIES_CONSTRAINED_HIGH_SPEED_VIDEO) {
if (Arrays.asList(map.getHighSpeedVideoFpsRanges()).contains(new Range<>(120, 120))) {
highSpeedFrameRate = new Range<>(120, 120);
} else {
highSpeedFrameRate = Collections.min(Arrays.asList(map.getHighSpeedVideoFpsRanges()), new CompareRange());
}
highSpeedSize = Collections.min(Arrays.asList(map.getHighSpeedVideoSizesFor(highSpeedFrameRate)), new CompareByArea());
One other weird thing: many phones, who's specs advertise high speed capabilities, report themselves as not able to record high speed.
OnePlus 3t crash:
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
Build fingerprint: 'OnePlus/OnePlus3/OnePlus3T:8.0.0/OPR1.170623.032/05171658:user/release-keys'
Revision: '0'
ABI: 'arm'
pid: 896, tid: 22783, name: C3Dev-0-ReqQueu >>> /system/bin/cameraserver <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x4
Cause: null pointer dereference
r0 00000001 r1 00000000 r2 00000002 r3 0000ec9e
r4 00000001 r5 ecdf9600 r6 0000f042 r7 00000002
r8 00000000 r9 ee7f96ac sl ecdb2f00 fp ecdb2f0c
ip 00000000 sp ede00718 lr f07a5aef pc f07a6838 cpsr 600f0030
backtrace:
#00 pc 000a7838 /system/lib/libcameraservice.so (_ZNSt3__113unordered_mapIKPK13native_handleyN7android13Camera3Device12HalInterface12BufferHasherENS7_16BufferComparatorENS_9allocatorINS_4pairIS4_yEEEEEixERS4_+159)
#01 pc 000a6aeb /system/lib/libcameraservice.so (_ZN7android13Camera3Device12HalInterface11getBufferIdERKPK13native_handlei+86)
#02 pc 000a69a9 /system/lib/libcameraservice.so (_ZN7android13Camera3Device12HalInterface17wrapAsHidlRequestEP23camera3_capture_requestPNS_8hardware6camera6device4V3_214CaptureRequestEPNSt3__16vectorIP13native_handleNSA_9allocatorISD_EEEE+352)
#03 pc 000a6d29 /system/lib/libcameraservice.so (_ZN7android13Camera3Device12HalInterface27processBatchCaptureRequestsERNSt3__16vectorIP23camera3_capture_requestNS2_9allocatorIS5_EEEEPj+144)
#04 pc 000a8357 /system/lib/libcameraservice.so (_ZN7android13Camera3Device13RequestThread17sendRequestsBatchEv+110)
#05 pc 000a8c0b /system/lib/libcameraservice.so (_ZN7android13Camera3Device13RequestThread10threadLoopEv+290)
#06 pc 0000d57b /system/lib/libutils.so (_ZN7android6Thread11_threadLoopEPv+270)
#07 pc 0004840f /system/lib/libc.so (_ZL15__pthread_startPv+22)
#08 pc 0001b54d /system/lib/libc.so (__start_thread+32)
This is due to the fact that High Speed Recording capabilities enabled but the configurations for 3rd party applications other than stock camera are locked by the Vendor parties.
Only option is to root the device and flash Custom or Stock ROM.
User complaint refs from OnePlus: