The sample below uses the Flutter camera & google_mlkit_barcode_scanning packages to detect a barcode.
The issue is that often it immediately returns an old result when the Scan Barcode button is pressed on the Home page while the camera is not pointing at any barcodes.
At around 22 seconds into this Screen Recording you can see there is no barcode in the camera preview but the previous barcode result is then quickly shown as detected.
Physical Testing Procedure:
Scan Barcode button => app navigates to BarcodeScanPage and starts image streamHome page.After a small number of iterations (usually on the second visit to BarcodeScanPage on my old Pixel 2, or after around a dozen visits on a more recent Samsung Galaxy s21) the BarcodeScanner returns the previously scanned result while the camera is still pointing AWAY from any barcodes!!!
void main() {
runApp(const MyApp());
}
class MyApp extends StatelessWidget {
const MyApp({Key? key}) : super(key: key);
@override
Widget build(BuildContext context) {
return MaterialApp(
title: 'BarcodeScanner Issue',
theme: ThemeData(primarySwatch: Colors.blue),
home: const HomePage(),
);
}
}
class HomePage extends StatelessWidget {
const HomePage({super.key});
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(title: const Text('Home')),
body: Center(
child: Column(
mainAxisAlignment: MainAxisAlignment.center,
children: <Widget>[
ElevatedButton(
onPressed: () {
Navigator.push(
context,
MaterialPageRoute(
builder: (context) => const BarcodeScanPage(),
),
);
},
child: const Text('Scan Barcode'),
),
],
),
),
);
}
}
class BarcodeScanPage extends StatefulWidget {
const BarcodeScanPage({super.key});
@override
State<BarcodeScanPage> createState() => BarcodeScanPageState();
}
class BarcodeScanPageState extends State<BarcodeScanPage> {
/// Used to control the cameras on the device.
late CameraController _controller;
/// The current selected camera to use.
late CameraDescription _currentCamera;
/// The scanned [Barcode] if any.
Barcode? _barcode;
/// Indicates if the async initialization is complete.
bool _initializing = true;
/// Indicates if an [InputImage] is currently being processed.
bool _isBusy = false;
/// ML-Kit Barcode Scanner.
final BarcodeScanner _barcodeScanner = BarcodeScanner();
@override
void initState() {
super.initState();
debugPrint('BarcodeScanPageState: initState()');
_initCamera();
}
@override
void dispose() {
debugPrint('BarcodeScanPageState: dispose()');
super.dispose();
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(title: const Text('Barcode Scan')),
body: Center(
child: _initializing
? const Text('Initializing...')
: _barcode == null
? CameraPreview(_controller)
: Column(
mainAxisSize: MainAxisSize.min,
children: [
const Text('Barcode detected'),
const SizedBox(height: 4.0),
Text(_barcode!.displayValue ?? 'Unknown'),
],
),
),
);
}
/// Initialize camera and controller, and start image stream.
Future _initCamera() async {
final cameras = await availableCameras();
_currentCamera = cameras.firstWhereOrNull((camera) {
return camera.lensDirection == CameraLensDirection.back &&
camera.sensorOrientation == 90;
}) ??
cameras.firstWhere((camera) {
return camera.lensDirection == CameraLensDirection.back;
});
_controller = CameraController(
_currentCamera,
ResolutionPreset.high,
enableAudio: false,
);
await _controller.initialize();
await _controller.startImageStream(_processCameraImage);
setState(() => _initializing = false);
}
/// Process a [CameraImage] into an [InputImage].
Future _processCameraImage(CameraImage image) async {
final WriteBuffer allBytes = WriteBuffer();
for (final Plane plane in image.planes) {
allBytes.putUint8List(plane.bytes);
}
final bytes = allBytes.done().buffer.asUint8List();
final Size imageSize = Size(
image.width.toDouble(),
image.height.toDouble(),
);
final imageRotation = InputImageRotationValue.fromRawValue(
_currentCamera.sensorOrientation,
);
if (imageRotation == null) return;
final inputImageFormat = InputImageFormatValue.fromRawValue(
image.format.raw,
);
if (inputImageFormat == null) return;
final planeData = image.planes.map(
(Plane plane) {
return InputImagePlaneMetadata(
bytesPerRow: plane.bytesPerRow,
height: plane.height,
width: plane.width,
);
},
).toList();
final inputImageData = InputImageData(
size: imageSize,
imageRotation: imageRotation,
inputImageFormat: inputImageFormat,
planeData: planeData,
);
final inputImage = InputImage.fromBytes(
bytes: bytes,
inputImageData: inputImageData,
);
_processInputImage(inputImage);
}
/// Process [InputImage] for [Barcode].
Future _processInputImage(InputImage inputImage) async {
if (_isBusy) return;
_isBusy = true;
final barcodes = await _barcodeScanner.processImage(inputImage);
if (barcodes.isNotEmpty) {
await _controller.stopImageStream();
await _controller.dispose();
setState(() => _barcode = barcodes.first);
}
_isBusy = false;
if (mounted) {
setState(() {});
}
}
}
pubspec.yaml dependencies:
camera: ^0.10.0+1
collection: ^1.16.0
google_mlkit_barcode_scanning: ^0.4.0
Below is a typical run log where this has occurred (Tested on an Android Pixel 2).
I have included debug statements in the init & dispose of the BarcodeScanPage to help follow the sequence of events.
BarcodeScanPage in this log the barcode scanned as expected.BarcodeScanPage in this log the BarcodeScanner returned the previously scanned Barcode while the camera was still pointing away from any barcodes.I/flutter (13032): BarcodeScanPageState: initState()
I/CameraManagerGlobal(13032): Connecting to camera service
W/Camera (13032): The selected imageFormatGroup is not supported by Android. Defaulting to yuv420
I/Camera (13032): startPreview
I/Camera (13032): CameraCaptureSession onConfigured
I/Camera (13032): Updating builder settings
D/Camera (13032): Updating builder with feature: ExposureLockFeature
D/Camera (13032): Updating builder with feature: ExposurePointFeature
D/Camera (13032): Updating builder with feature: ZoomLevelFeature
D/Camera (13032): Updating builder with feature: AutoFocusFeature
D/Camera (13032): Updating builder with feature: NoiseReductionFeature
I/Camera (13032): updateNoiseReduction | currentSetting: fast
D/Camera (13032): Updating builder with feature: FocusPointFeature
D/Camera (13032): Updating builder with feature: ResolutionFeature
D/Camera (13032): Updating builder with feature: SensorOrientationFeature
D/Camera (13032): Updating builder with feature: FlashFeature
D/Camera (13032): Updating builder with feature: ExposureOffsetFeature
D/Camera (13032): Updating builder with feature: FpsRangeFeature
I/Camera (13032): refreshPreviewCaptureSession
W/Gralloc4(13032): allocator 3.x is not supported
W/Gralloc3(13032): allocator 3.x is not supported
I/Camera (13032): startPreviewWithImageStream
I/Camera (13032): CameraCaptureSession onConfigured
I/Camera (13032): Updating builder settings
D/Camera (13032): Updating builder with feature: ExposureLockFeature
D/Camera (13032): Updating builder with feature: ExposurePointFeature
D/Camera (13032): Updating builder with feature: ZoomLevelFeature
D/Camera (13032): Updating builder with feature: AutoFocusFeature
D/Camera (13032): Updating builder with feature: NoiseReductionFeature
I/Camera (13032): updateNoiseReduction | currentSetting: fast
D/Camera (13032): Updating builder with feature: FocusPointFeature
D/Camera (13032): Updating builder with feature: ResolutionFeature
D/Camera (13032): Updating builder with feature: SensorOrientationFeature
D/Camera (13032): Updating builder with feature: FlashFeature
D/Camera (13032): Updating builder with feature: ExposureOffsetFeature
D/Camera (13032): Updating builder with feature: FpsRangeFeature
I/Camera (13032): refreshPreviewCaptureSession
I/Camera (13032): CameraCaptureSession onClosed
W/System (13032): A resource failed to call release.
D/TransportRuntime.JobInfoScheduler(13032): Scheduling upload for context TransportContext(cct, VERY_LOW, MSRodHRwczovL2ZpcmViYXNlbG9nZ2luZy5nb29nbGVhcGlzLmNvbS92MGNjL2xvZy9iYXRjaD9mb3JtYXQ9anNvbl9wcm90bzNc) with jobId=1208757821 in 86400000ms(Backend next call timestamp 0). Attempt 1
I/ra_stream_issu(13032): Waiting for a blocking GC ClassLinker
I/DynamiteModule(13032): Considering local module com.google.mlkit.dynamite.barcode:10000 and remote module com.google.mlkit.dynamite.barcode:0
I/DynamiteModule(13032): Selected local version of com.google.mlkit.dynamite.barcode
I/TetheringManager(13032): registerTetheringEventCallback:au.com.soundconception.camera_stream_issue
D/TransportRuntime.SQLiteEventStore(13032): Storing event with priority=VERY_LOW, name=FIREBASE_ML_SDK for destination cct
D/TransportRuntime.JobInfoScheduler(13032): Upload for context TransportContext(cct, VERY_LOW, MSRodHRwczovL2ZpcmViYXNlbG9nZ2luZy5nb29nbGVhcGlzLmNvbS92MGNjL2xvZy9iYXRjaD9mb3JtYXQ9anNvbl9wcm90bzNc) is already scheduled. Returning...
D/TransportRuntime.SQLiteEventStore(13032): Storing event with priority=DEFAULT, name=FIREBASE_ML_SDK for destination cct
D/TransportRuntime.JobInfoScheduler(13032): Scheduling upload for context TransportContext(cct, DEFAULT, MSRodHRwczovL2ZpcmViYXNlbG9nZ2luZy5nb29nbGVhcGlzLmNvbS92MGNjL2xvZy9iYXRjaD9mb3JtYXQ9anNvbl9wcm90bzNc) with jobId=1203777084 in 259261ms(Backend next call timestamp 1661840669048). Attempt 1
I/ra_stream_issu(13032): Background concurrent copying GC freed 6988(483KB) AllocSpace objects, 22(28MB) LOS objects, 23% free, 79MB/103MB, paused 353us total 147.579ms
D/TransportRuntime.SQLiteEventStore(13032): Storing event with priority=VERY_LOW, name=FIREBASE_ML_SDK for destination cct
D/TransportRuntime.JobInfoScheduler(13032): Upload for context TransportContext(cct, VERY_LOW, MSRodHRwczovL2ZpcmViYXNlbG9nZ2luZy5nb29nbGVhcGlzLmNvbS92MGNjL2xvZy9iYXRjaD9mb3JtYXQ9anNvbl9wcm90bzNc) is already scheduled. Returning...
I/tflite (13032): Initialized TensorFlow Lite runtime.
I/native (13032): I0830 15:50:09.888608 13236 oned_decoder_client.cc:685] barhopper::deep_learning::OnedDecoderClient is created successfully.
E/libc (13032): Access denied finding property "ro.hardware.chipname"
D/TransportRuntime.SQLiteEventStore(13032): Storing event with priority=VERY_LOW, name=FIREBASE_ML_SDK for destination cct
D/TransportRuntime.JobInfoScheduler(13032): Upload for context TransportContext(cct, VERY_LOW, MSRodHRwczovL2ZpcmViYXNlbG9nZ2luZy5nb29nbGVhcGlzLmNvbS92MGNjL2xvZy9iYXRjaD9mb3JtYXQ9anNvbl9wcm90bzNc) is already scheduled. Returning...
D/TransportRuntime.SQLiteEventStore(13032): Storing event with priority=VERY_LOW, name=FIREBASE_ML_SDK for destination cct
D/TransportRuntime.JobInfoScheduler(13032): Upload for context TransportContext(cct, VERY_LOW, MSRodHRwczovL2ZpcmViYXNlbG9nZ2luZy5nb29nbGVhcGlzLmNvbS92MGNjL2xvZy9iYXRjaD9mb3JtYXQ9anNvbl9wcm90bzNc) is already scheduled. Returning...
I/ra_stream_issu(13032): Background concurrent copying GC freed 4314(1147KB) AllocSpace objects, 86(103MB) LOS objects, 23% free, 79MB/103MB, paused 194us total 131.504ms
I/Camera (13032): startPreview
I/Camera (13032): CameraCaptureSession onConfigured
I/Camera (13032): Updating builder settings
D/Camera (13032): Updating builder with feature: ExposureLockFeature
D/Camera (13032): Updating builder with feature: ExposurePointFeature
D/Camera (13032): Updating builder with feature: ZoomLevelFeature
D/Camera (13032): Updating builder with feature: AutoFocusFeature
D/Camera (13032): Updating builder with feature: NoiseReductionFeature
I/Camera (13032): updateNoiseReduction | currentSetting: fast
D/Camera (13032): Updating builder with feature: FocusPointFeature
D/Camera (13032): Updating builder with feature: ResolutionFeature
D/Camera (13032): Updating builder with feature: SensorOrientationFeature
D/Camera (13032): Updating builder with feature: FlashFeature
D/Camera (13032): Updating builder with feature: ExposureOffsetFeature
D/Camera (13032): Updating builder with feature: FpsRangeFeature
I/Camera (13032): refreshPreviewCaptureSession
I/Camera (13032): dispose
I/Camera (13032): close
I/Camera (13032): open | onClosed
I/flutter (13032): BarcodeScanPageState: dispose()
I/flutter ( 8761): is subscribed? false
I/flutter (13032): BarcodeScanPageState: initState()
I/Camera (13032): close
W/Camera (13032): The selected imageFormatGroup is not supported by Android. Defaulting to yuv420
I/Camera (13032): startPreview
I/Camera (13032): CameraCaptureSession onConfigured
I/Camera (13032): Updating builder settings
D/Camera (13032): Updating builder with feature: ExposureLockFeature
D/Camera (13032): Updating builder with feature: ExposurePointFeature
D/Camera (13032): Updating builder with feature: ZoomLevelFeature
D/Camera (13032): Updating builder with feature: AutoFocusFeature
D/Camera (13032): Updating builder with feature: NoiseReductionFeature
I/Camera (13032): updateNoiseReduction | currentSetting: fast
D/Camera (13032): Updating builder with feature: FocusPointFeature
D/Camera (13032): Updating builder with feature: ResolutionFeature
D/Camera (13032): Updating builder with feature: SensorOrientationFeature
D/Camera (13032): Updating builder with feature: FlashFeature
D/Camera (13032): Updating builder with feature: ExposureOffsetFeature
D/Camera (13032): Updating builder with feature: FpsRangeFeature
I/Camera (13032): refreshPreviewCaptureSession
I/Camera (13032): startPreviewWithImageStream
I/Camera (13032): CameraCaptureSession onClosed
I/Camera (13032): CameraCaptureSession onConfigured
I/Camera (13032): Updating builder settings
D/Camera (13032): Updating builder with feature: ExposureLockFeature
D/Camera (13032): Updating builder with feature: ExposurePointFeature
D/Camera (13032): Updating builder with feature: ZoomLevelFeature
D/Camera (13032): Updating builder with feature: AutoFocusFeature
D/Camera (13032): Updating builder with feature: NoiseReductionFeature
I/Camera (13032): updateNoiseReduction | currentSetting: fast
D/Camera (13032): Updating builder with feature: FocusPointFeature
D/Camera (13032): Updating builder with feature: ResolutionFeature
D/Camera (13032): Updating builder with feature: SensorOrientationFeature
D/Camera (13032): Updating builder with feature: FlashFeature
D/Camera (13032): Updating builder with feature: ExposureOffsetFeature
D/Camera (13032): Updating builder with feature: FpsRangeFeature
I/Camera (13032): refreshPreviewCaptureSession
D/TransportRuntime.SQLiteEventStore(13032): Storing event with priority=DEFAULT, name=FIREBASE_ML_SDK for destination cct
D/TransportRuntime.JobInfoScheduler(13032): Upload for context TransportContext(cct, DEFAULT, MSRodHRwczovL2ZpcmViYXNlbG9nZ2luZy5nb29nbGVhcGlzLmNvbS92MGNjL2xvZy9iYXRjaD9mb3JtYXQ9anNvbl9wcm90bzNc) is already scheduled. Returning...
I/Camera (13032): startPreview
I/Camera (13032): CameraCaptureSession onConfigured
I/Camera (13032): Updating builder settings
D/Camera (13032): Updating builder with feature: ExposureLockFeature
D/Camera (13032): Updating builder with feature: ExposurePointFeature
D/Camera (13032): Updating builder with feature: ZoomLevelFeature
D/Camera (13032): Updating builder with feature: AutoFocusFeature
D/Camera (13032): Updating builder with feature: NoiseReductionFeature
I/Camera (13032): updateNoiseReduction | currentSetting: fast
D/Camera (13032): Updating builder with feature: FocusPointFeature
D/Camera (13032): Updating builder with feature: ResolutionFeature
D/Camera (13032): Updating builder with feature: SensorOrientationFeature
D/Camera (13032): Updating builder with feature: FlashFeature
D/Camera (13032): Updating builder with feature: ExposureOffsetFeature
D/Camera (13032): Updating builder with feature: FpsRangeFeature
I/Camera (13032): refreshPreviewCaptureSession
I/Camera (13032): dispose
I/Camera (13032): close
I/Camera (13032): open | onClosed
W/System (13032): A resource failed to call release.
I/flutter ( 8761): wasAuthenticated: true, isAuthenticated true
I/flutter ( 8761): wasAuthenticated: true, isAuthenticated true
I/flutter ( 8761): wasAuthenticated: true, isAuthenticated true
I have tried clearing the image cache between visits to the BarcodeScanPage with the following code, but it does not change the result:
imageCache.clearLiveImages();
imageCache.clear();
I have also tried adding a global previousCameraImage variable and compared the current image byte data to the previous image byte data to see if the same image is being served, by making the following code additions:
...
CameraImage? previousCameraImage;
class BarcodeScanPage extends StatefulWidget {
...
Future _processCameraImage(CameraImage image) async {
final isPreviousCameraImage =
image.planes.firstWhereIndexedOrNull((index, plane) {
return !listEquals(
plane.bytes,
previousCameraImage?.planes[index].bytes,
);
}) ==
null;
debugPrint(
'_processCameraImage isPreviousCameraImage: $isPreviousCameraImage',
);
if (isPreviousCameraImage) return;
final WriteBuffer allBytes = WriteBuffer();
...
NOTE: The CameraImage class does not implement an equality operator, so I believe the above is a reasonable alternative, but am happy to be corrected!
When I run the test with this code change isPreviousCameraImage never resolves to true and the main issue still occurs.
This result, combined with the fact that the Screen Recording shows that the camera is not pointing at a barcode when the second "barcode detection" occurs, would suggest that the issue may be with the google_mlkit_barcode_scanning package.
If nobody has any solutions, I would also appreciate any suggestions for further investigation. I will also be posting this as an issue to the google_mlkit_barcode_scanning package developer shortly if I'm unable to find a solution.
The actual issue is the CameraController.
I checked by converting the CameraImage that had been used each time a successful BarcodeScanner result was obtained, into a png, and displayed it next to the successful barcode result.
From this I could see that occasionally when you call _controller.startImageStream(_processCameraImage) the very first image returned is sometimes identical to the last image returned when you previously stopped the stream with _controller.stopImageStream(). In other words, the stream is not always cleared when it is restarted.
A simple, safe and quick solution is to always discard the very first image delivered after calling _controller.startImageStream(_processCameraImage).
class BarcodeScanPageState extends State<BarcodeScanPage> {
...
// Indicates whether the first `CameraImage` from the Image Stream has been discarded.
bool _firstImageDiscarded = false;
...
/// Process a [CameraImage] into an [InputImage].
Future _processCameraImage(CameraImage image) async {
// Check if the first CameraImage has been discarded
if (!_firstImageDiscarded) {
// Update the boolean flag to true.
_firstImageDiscarded = true;
// Return immediately without processing the image, effectively discarding it.
return;
}
...
}
...
}