Search code examples
swiftavfoundationcidetector

CIDetector face detection in real-time However memory consumption increases linearly how to avoid this issue?


I have a question how to correctly call CIDetector correctly I'm trying to run the face detection in real-time this works very well. However the memory consumption of the app increases linearly with time how you can see in the image below I'm thinking this is due to objects being created but they're not released can anyone advise how to do it correctly.

I have pinpointed the issue down to this function as every time it's invoked memory increases linearly when it terminated it quickly drops down to almost 80 MB instead of 11 GB rising also check for memory leaks however none were found.

My target development platform is Mac OS I'm trying to extractthe mouth position from the CA detector and then use it to compute a Delta in the mouse function for a Game.

I also Looked that this post however I have tried their approach but it did not work for me CIDetector isn't releasing memory

enter image description here

 fileprivate func faceDetection(){

    // setting up dispatchQueue
    dispatchQueue.async {

        //  checking if sample buffer  is equal to nil if not assign its value to sample
        if let sample = self.sampleBuffers {


            //   if allfeatures is not equal to nil. if yes assign allfeatures to features otherwise return
            guard let features = self.allFeatures(sample: sample) else { return }

            // loop to cycle through all features
            for  feature in features {

                // checks if the feature is a CIFaceFeature if yes assign feature to face feature and go on.
                if let faceFeature = feature as? CIFaceFeature {


                    if !self.hasEnded {

                        if self.calX.count > 30 {
                            self.sens.append((self.calX.max()! - self.calX.min()!))
                            self.sens.append((self.calY.max()! - self.calY.min()!))
                            print((self.calX.max()! - self.calX.min()!))
                            self.hasEnded = true
                        } else {
                            self.calX.append(faceFeature.mouthPosition.x)
                            self.calY.append(faceFeature.mouthPosition.y)

                        }

                    } else {
                        self.mouse(position:  CGPoint(x: (faceFeature.mouthPosition.x  - 300 ) * 2, y: (faceFeature.mouthPosition.y + 20 ) * 2), faceFeature: faceFeature)

                    }
                }
            }
        }

        if !self.faceTrackingEnds {
            self.faceDetection()
        }
    }
}

Solution

  • This problem was caused by repeatedly calling the function without waiting for its completion the fix was implementing a dispatch group and then calling the function on its completion like this Now the CIdetector runs comfortably at 200 MB memory

        fileprivate func faceDetection(){
    
           let group = DispatchGroup()
           group.enter()
    
           // setting up dispatchQueue
           dispatchQueue.async {
    
            //  checking if sample buffer  is equal to nil if not assign its value to sample
            if let sample = self.sampleBuffers {
    
    
                //   if allfeatures is not equal to nil. if yes assign allfeatures to features otherwise return
                guard let features = self.allFeatures(sample: sample) else { return }
    
                // loop to cycle through all features
                for  feature in features {
    
                    // checks if the feature is a CIFaceFeature if yes assign feature to face feature and go on.
                    if let faceFeature = feature as? CIFaceFeature {
    
    
                            self.mouse(position: faceFeature.mouthPosition, faceFeature: faceFeature)
    
                    }
                }
            }
            group.leave()
        }
        group.notify(queue: .main) {
            if !self.faceTrackingEnds {
                self.faceDetection()
            }
        }
    
    }