Search code examples
iosswiftgrand-central-dispatchsemaphoredispatch

How to manage download queue?


I am taking user input to download files from the server. The downloading task may include requesting web services.

I am expecting something like this:

1) Whenever the user selects a file to download or requesting for web service, then it should be treated as one block of operation or task and should go in the queue which will be managed globally at the app level.
2) At the same time if the queue is empty then it should automatically start executing the current task.
3) If queue contains any operation then it should execute all old operation in synchronously then execute the last one.

Can any one suggest how this can be done by the optimized way?

Take a look what I tried:

class func downloadChaptersFromDownloadQueue() {

    let gbm = GlobalMethods()

    for chapterDetail in gbm.downloadOpertationQueue.array.enumerated() {

        if chapterDetail.element.chapterdata.state == .non || chapterDetail.element.chapterdata.state == .paused || chapterDetail.element.chapterdata.state == .downloading {

            gbm.downloadOpertationQueue[chapterDetail.offset].chapterdata.state = .downloading
                let s = DispatchSemaphore(value: 0)

                self.downloadActivty(courseId: chapterDetail.element.courseId, mod: chapterDetail.element.chapterdata, selectedIndexpath: chapterDetail.element.cellForIndexpath, success: { (result) in

                    if (result) {
                        if (WC_SQLite.shared.updateChapterState(courseId: chapterDetail.element.courseId, chapterId: chapterDetail.element.chapterdata.id, state: .downloaded)) {
                            s.signal()

                            gbm.downloadOpertationQueue[chapterDetail.offset].chapterdata.state = .downloaded
                            NotificationCenter.default.post(name: NSNotification.Name(("DownloadChapter")), object: self, userInfo: ["progress": 1.0, "notifIdentifier":(chapterDetail.element.cellForIndexpath)])
                        }
                        else {
                            s.signal()
                            gbm.downloadOpertationQueue[chapterDetail.offset].chapterdata.state = .non
                            NotificationCenter.default.post(name: NSNotification.Name(("DownloadChapter")), object: self, userInfo: ["progress": -1.0, "notifIdentifier":(chapterDetail.element.cellForIndexpath)])
                        }
                    }
                    else {
                        _ = WC_SQLite.shared.updateChapterState(courseId: chapterDetail.element.courseId, chapterId: chapterDetail.element.chapterdata.id, state: .non)

                        s.signal()

                        gbm.downloadOpertationQueue[chapterDetail.offset].chapterdata.state = .non
                        NotificationCenter.default.post(name: NSNotification.Name(("DownloadChapter")), object: self, userInfo: ["progress": -1.0, "notifIdentifier":(chapterDetail.element.cellForIndexpath)])
                    }
                })
                s.wait()
        }
    }
}

Solution

  • Create an async queue. First use a dispatch group to track how many requests are completed and get notified when all are finished (completely async).

    Next, enqueue ALL your requests. Each request should have a unique identifier so you know which request completed or failed (the chapterId & pageNumber in your case should be enough).

    Execute all the requests at once (again it is async) and you will be notified when each one completes (on the main queue through your completion block). The completion block should be called with all the requests responses and their unique identifiers.

    Example:

    class NetworkResponse {
        let data: Data?
        let response: URLResponse?
        let error: Error?
    
        init(data: Data?, response: URLResponse?, error: Error?) {
            self.data = data
            self.response = response
            self.error = error
        }
    }
    
    
    class NetworkQueue {
        static let instance = NetworkQueue()
        private let group = DispatchGroup()
        private let lock = DispatchSemaphore(value: 0)
        private var tasks = Array<URLSessionDataTask>()
        private var responses = Dictionary<String, NetworkResponse>()
    
        private init() {
    
        }
    
        public func enqueue(request: URLRequest, requestID: String) {
    
            //Create a task for each request and add it to the queue (we do not execute it yet). Every request that is created, we enter our group.
    
            self.group.enter();
            let task = URLSession.shared.dataTask(with: request) { (data, response, error) in
    
                //Only one thread can modify the array at any given time.
                objc_sync_enter(self)
                self.responses.updateValue(NetworkResponse(data: data, response: response, error: error), forKey: requestID)
                objc_sync_exit(self)
    
                //Once the request is complete, it needs to leave the group.
                self.group.leave()
            }
    
            //Add each task to the queue.
            self.tasks.append(task)
        }
    
        public func execute(completion: @escaping (_ responses: Dictionary<String, NetworkResponse>) -> Void) {
    
            //Get notified on the main queue when every single request is completed (they all happen asynchronously, but we get one notification)
    
            self.group.notify(queue: DispatchQueue.main) { 
    
                //Call our completion block with all the responses. Might be better to use a sorted dictionary or something here so that the responses are in order.. but for now, a Dictionary with unique identifiers will be fine.
                completion(self.responses)
            }
    
            //Execute every task in the queue.
            for task in self.tasks {
                task.resume()
            }
    
            //Clear all executed tasks from the queue.
            self.tasks.removeAll()
        }
    }
    

    EDIT (Using your own code):

    class func downloadChaptersFromDownloadQueue() {
    
    
        let gbm = GlobalMethods()
        let group = DispatchGroup()
        let lock = NSLock()
    
        //Get notified when ALL tasks have completed.
        group.notify(queue: DispatchQueue.main) {
            print("FINISHED ALL TASKS -- DO SOMETHING HERE")
        }
    
        //Initially enter to stall the completion
        group.enter()
    
        defer {
            group.leave() //Exit the group to complete the enqueueing.
        }
    
        for chapterDetail in gbm.downloadOpertationQueue.array.enumerated() {
    
            if chapterDetail.element.chapterdata.state == .non || chapterDetail.element.chapterdata.state == .paused || chapterDetail.element.chapterdata.state == .downloading {
    
                gbm.downloadOpertationQueue[chapterDetail.offset].chapterdata.state = .downloading
    
                //Enter the group for each downloadOperation
                group.enter()
    
                self.downloadActivty(courseId: chapterDetail.element.courseId, mod: chapterDetail.element.chapterdata, selectedIndexpath: chapterDetail.element.cellForIndexpath, success: { (result) in
    
                    lock.lock()
                    defer {
                        group.leave() //Leave the group when each downloadOperation is completed.
                    }
    
                    if (result) {
                        if (WC_SQLite.shared.updateChapterState(courseId: chapterDetail.element.courseId, chapterId: chapterDetail.element.chapterdata.id, state: .downloaded)) {
    
                            gbm.downloadOpertationQueue[chapterDetail.offset].chapterdata.state = .downloaded
                            lock.unlock()
    
                            NotificationCenter.default.post(name: NSNotification.Name(("DownloadChapter")), object: self, userInfo: ["progress": 1.0, "notifIdentifier":(chapterDetail.element.cellForIndexpath)])
                        }
                        else {
                            gbm.downloadOpertationQueue[chapterDetail.offset].chapterdata.state = .non
                            lock.unlock()
    
                            NotificationCenter.default.post(name: NSNotification.Name(("DownloadChapter")), object: self, userInfo: ["progress": -1.0, "notifIdentifier":(chapterDetail.element.cellForIndexpath)])
                        }
                    }
                    else {
                        _ = WC_SQLite.shared.updateChapterState(courseId: chapterDetail.element.courseId, chapterId: chapterDetail.element.chapterdata.id, state: .non)
    
                        gbm.downloadOpertationQueue[chapterDetail.offset].chapterdata.state = .non
                        lock.unlock()
    
                        NotificationCenter.default.post(name: NSNotification.Name(("DownloadChapter")), object: self, userInfo: ["progress": -1.0, "notifIdentifier":(chapterDetail.element.cellForIndexpath)])
                    }
                })
            }
        }
    }
    

    Again, this is asynchronous because you don't want the user waiting forever to download 100 pages..