Buffer point issue and main thread is blocked using CoreML

Question

I am using CoreML to remove the background in a given image and I have 2 major problems with it.

The model I embedded in my Xcode is DeepLabV3.mlmodel. This is the entire class that returns the final image: RemoveBackgroundCoreML_Final.swift which I just imported into my source code. It works well in terms of doing its job. However, I have a warning in this swift class at line 107:

let data = Data(buffer: UnsafeBufferPointer(start: &cubeRGB, count: cubeRGB.count))

Initialization of 'UnsafeMutableBufferPointer' results in a dangling buffer pointer

The worse case is, this line of the code crashes the production app according to my crash logs from the App Store for some devices, even though I myself don't experience any crash in the development environment.

This is how i call removeBackground method in above Swift class from my code:

let finalImage = image.removeBackground(returnResult: .finalImage)

Another issue is, above line returns finalImage but it also blocks the main thread in the app which has some UI animation just started to run previously to show some process (which I suspect, could be related to buffer pointer).

I hope anyone who has experience with CoreML or threading can help me and I too much appreciate it. Please ask, if further clarification needed. Thank you in advance.

Answer 1

Since you're not using cubeRGB after making the pointer to it, the Swift optimizer will deallocate it immediately afterwards. As this is an optimizer thing, you won't get this bug in debug mode, only in release mode.

Data(buffer:...) will make a copy of the array, so that's good. You just need to make the array stays alive until it's done copying. This can be done with withExtendedLifetime, something like:

let data = withExtendedLifetime(cubeRGB) {
    return Data(buffer: UnsafeBufferPointer(start: &cubeRGB, count: cubeRGB.count))
}

I've probably got the syntax wrong but something like that.