Search code examples
flutterdartcrashout-of-memory

Flutter: Async call of native C code from Dart (via ffi) cause memory leak

Randomly I'm getting empty screen of my Flutter app with apparently out of memory error. See the DEBUG CONSOLE log below.

Here is my Dart DevTools Memory view during the crash - as you can see the memory consumption stays stable during the entire session:

https://i.sstatic.net/qGwgl.png

I'm wondering if there is any other memory type that I should check?

BTW, the app doesn't actually crash, but shows empty (background color) screen. It reacts to taps somehow - because I can see my debug prints from some of my widgets. It reacts to taps but doesn't displays anything - the screen is empty.

DEBUG CONSOLE log while app is in the state of showing empty screen:

libc    ( 8725): pthread_create failed: couldn't mprotect R+W 1028096-byte thread mapping region: Out of memory
W/Adreno-GSL( 8725): <sharedmem_gpuobj_alloc:2706>: sharedmem_gpumem_alloc: mmap failed errno 12 Out of memory
E/Adreno-GSL( 8725): <gsl_memory_alloc_pure:2297>: GSL MEM ERROR: kgsl_sharedmem_alloc ioctl failed.
W/Adreno-GSL( 8725): <sharedmem_gpuobj_alloc:2706>: sharedmem_gpumem_alloc: mmap failed errno 12 Out of memory
E/Adreno-GSL( 8725): <gsl_memory_alloc_pure:2297>: GSL MEM ERROR: kgsl_sharedmem_alloc ioctl failed.
W/Adreno-GSL( 8725): <sharedmem_gpuobj_alloc:2706>: sharedmem_gpumem_alloc: mmap failed errno 12 Out of memory
E/Adreno-GSL( 8725): <gsl_memory_alloc_pure:2297>: GSL MEM ERROR: kgsl_sharedmem_alloc ioctl failed.
W/Adreno-GSL( 8725): <sharedmem_gpuobj_alloc:2706>: sharedmem_gpumem_alloc: mmap failed errno 12 Out of memory
E/Adreno-GSL( 8725): <gsl_memory_alloc_pure:2297>: GSL MEM ERROR: kgsl_sharedmem_alloc ioctl failed.
W/Adreno-GSL( 8725): <sharedmem_gpuobj_alloc:2706>: sharedmem_gpumem_alloc: mmap failed errno 12 Out of memory
E/Adreno-GSL( 8725): <gsl_memory_alloc_pure:2297>: GSL MEM ERROR: kgsl_sharedmem_alloc ioctl failed.
W/Adreno-GSL( 8725): <sharedmem_gpuobj_alloc:2706>: sharedmem_gpumem_alloc: mmap failed errno 12 Out of memory
E/Adreno-GSL( 8725): <gsl_memory_alloc_pure:2297>: GSL MEM ERROR: kgsl_sharedmem_alloc ioctl failed.
W/Adreno-GSL( 8725): <sharedmem_gpuobj_alloc:2706>: sharedmem_gpumem_alloc: mmap failed errno 12 Out of memory
E/Adreno-GSL( 8725): <gsl_memory_alloc_pure:2297>: GSL MEM ERROR: kgsl_sharedmem_alloc ioctl failed.
...
...
I/Surface ( 8725): opservice is null false

flutter doctor -v

[√] Flutter (Channel stable, 1.22.5, on Microsoft Windows [Version 10.0.18363.1256], locale en-US)
    • Flutter version 1.22.5 at C:\sdk\flutter
    • Framework revision 7891006299 (5 weeks ago), 2020-12-10 11:54:40 -0800
    • Engine revision ae90085a84
    • Dart version 2.10.4

[√] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at C:\Users\esikerin\AppData\Local\Android\sdk
    • Platform android-30, build-tools 30.0.2
    • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)
    • All Android licenses accepted.

[!] Android Studio (version 4.0)
    • Android Studio at C:\Program Files\Android\Android Studio
    X Flutter plugin not installed; this adds Flutter specific functionality.
    X Dart plugin not installed; this adds Dart specific functionality.
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)

[√] VS Code (version 1.52.1)
    • VS Code at C:\Users\esikerin\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.18.1

[√] Connected device (1 available)
    • GM1900 (mobile) • ff184dad • android-arm64 • Android 10 (API 29)

UPDATE:

To give a little more background. I started getting this issue after I implemented running native C-code (via ffi) asynchronously from my Dart code in the following manner.

I create list of Uint8 in Dart and pass it to my C-code. C-code will start several parallel threads and return to Dart. Length of the list is equal to the number of threads. Then I check the list in my Dart code periodically to know when all C-thread finished. Obviously C-threads will set corresponding list element to 1 when finished. This way I can wait parallel C-threads to finish from Dart code without blocking my main UI thread. BTW, this waiting loop is only running when I'm waiting C-code to finish - so I don't run it during the entire life-cycle of the application.

  var threadStatusesPtr = xffi.allocate<ffi.Uint8>(count: _numThreads);
  var threadStatuses = threadStatusesPtr.asTypedList(_numThreads);
  for (int i=0; i<threadStatuses.length; i++) {
    threadStatuses[i] = 0;
  }

  // run C-function that starts parallel threads and returns
  // each C-thread when finished writes 1 to corresponding threadStatuses array element
  // so when all C-threads finished all elements in threadStatuses list should be non-zero
  _applyFiltersNative(_numThreads, threadStatusesPtr, ...);

  // waiting for all C-threads to finish
  while (threadStatuses.any((status) => status == 0)) {
    await Future.delayed(Duration(milliseconds: 10));
  }

I'm wondering if there is any potential issue, why I should not access the same list from Dart code that is being modified in parallel C-threads.

P.S.: Previously when I was running my C-code synchronously by waiting all threads to finish (pthread_join) before returning to Dart I never encountered this issue.

Solution

  • I've found the root cause of the problem.

    By default pthread_create() creates joinable threads. That means you have to join them with pthread_join() - overwise they will never release resources allocated. This is exactly what happened in my case - completed threads never released memory allocated, thus I encountered "Out of Memory" issue after some time. And that's why I've never seen this problem with synchronous call - because I waited all my threads with join.

    If you want to call your C-func asynchronously from Dart code you don't want to wait your threads with pthead_join() - you want to create thread(s) and return to Dart. In this case you have to create detached thread(s) - i.e. call pthread_detach() after pthread_create() - this way your threads won't hold any resources when finished.

    The only remaining question is: why I was not able to see any memory leak in Dart DevTools? If I saw the memory leak, I would probably fixed this issue much earlier, before posting anything to Stack Overflow :)