Making multiple asynchronous HTTP requests in succession and writing with Realm

Question

I'm currently using Alamofire for requesting data and writing to disk with Realm. Specifically, I am fetching 24 source URLS from a Facebook Graph GET request and then making 24 separate requests to retrieve the data for each image. Once the data is retrieved, I am writing to disk with Realm.

here is how I am fetching the 24 sources:

FBAPI

Alamofire.request(.GET, FBPath.photos, parameters: params).responseJSON { response in
            guard response.result.error == nil else {
                print("error calling GET on \(FBPath.photos)")
                print(response.result.error!)
                completion(latestDate: nil, photosCount: 0, error: response.result.error)
                return
            }
            if let value = response.result.value {
                let json = JSON(value)
                if let photos = json[FBResult.data].array {
                    for result in photos {
                        let manager = PTWPhotoManager()
                        manager.downloadAndSaveJsonData(result)
                    }

As you can see, I have a for loop iterating through each JSON containing the source url for the photo's image in which I then make another network request for each url, like so:

Manager

func downloadAndSaveJsonData(photoJSON : JSON) {

        let source = photoJSON[FBResult.source].string
        let id = photoJSON[FBResult.id].string
        let created_time = photoJSON[FBResult.date.createdTime].string
        let imageURL = NSURL(string: source!)

        print("image requested")
        Alamofire.request(.GET, imageURL!).response() {
            (request, response, data, error) in
            if (error != nil) {
                print(error?.localizedDescription)
            }
            else {
                print("image response")
                let photo = PTWPhoto()
                photo.id = id
                photo.sourceURL = source
                photo.imageData = data
                photo.createdTime = photo.createdTimeAsDate(created_time!)
                let realm = try! Realm()
                try! realm.write {
                    realm.add(photo)
                }
                print("photo saved")
            }
        }
    }

There seems to be a very long delay between when each image's data is requested and when I receive a response, and it also does not appear to be asynchronous. Is this a threading issue or is there a more efficient way to request an array of data like this? It should also be noted that I am making this network request from the Apple Watch itself.

Answer 1

These requests will happen mostly asynchronous as you wish. But there is some synchronization happening, you might been not aware of:

1. The response closures for Alamofire are dispatched to the main thread. So your network responses competes against any UI updates you do.
2. Realm write transactions are synchronous and exclusive, which is enforced via locks which will block the thread where they are executed on.

In combination this both means that you will block the main thread as long as the network requests succeed and keep coming, which would also render your app unresponsive.

I'd recommend a different attempt. You can use GCD's dispatch groups to synchronize different asynchronous tasks.

In the example below, the objects are all kept in memory until they are all downloaded.

A further improvement could it be to write the downloaded data onto disk instead and store just the path to the file in the Realm object. (There are plenty of image caching libraries, which can easily assist you with that.)

If you choose a path, which depends only on the fields of PWTPhoto (or properties of the data, you can get through a quick HEAD request), then you can check first whether this path exists already locally before downloading the file again. By doing that you save traffic when updating the photos or when not all photos could been successfully downloaded on the first attempt. (e.g. app is force-closed by the user, crashed, device is shutdown)

class PTWPhotoManager {

    static func downloadAllPhotos(params: [String : AnyObject], completion: (latestDate: NSDate?, photosCount: NSUInteger, error: NSError?)) {
        Alamofire.request(.GET, FBPath.photos, parameters: params).responseJSON { response in
            guard response.result.error == nil else {
                print("error calling GET on \(FBPath.photos)")
                print(response.result.error!)
                completion(latestDate: nil, photosCount: 0, error: response.result.error)
                return
            }
            if let value = response.result.value {
                let json = JSON(value)
                if let photos = json[FBResult.data].array {
                    let group = dispatch_group_create()
                    var persistablePhotos = [PTWPhoto](capacity: photos.count)
                    let manager = PTWPhotoManager()
                    for result in photos {
                        dispatch_group_enter(group)
                        let request = manager.downloadAndSaveJsonData(result) { photo, error in
                            if let photo = photo {
                                persistablePhotos.add(photo)
                                dispatch_group_leave(group)
                            } else {
                                completion(latestDate: nil, photosCount: 0, error: error!)
                            }
                        }
                    }

                    dispatch_group_notify(group, dispatch_get_main_queue()) {
                        let realm = try! Realm()
                        try! realm.write {
                            realm.add(persistablePhotos)
                        }
                        let latestDate = …
                        completion(latestDate: latestDate, photosCount: persistablePhotos.count, error: nil)
                    }
                }
            }
        }
    }

    func downloadAndSaveJsonData(photoJSON: JSON, completion: (PTWPhoto?, NSError?) -> ()) -> Alamofire.Request {
        let source = photoJSON[FBResult.source].string
        let id = photoJSON[FBResult.id].string
        let created_time = photoJSON[FBResult.date.createdTime].string
        let imageURL = NSURL(string: source!)

        print("image requested")
        Alamofire.request(.GET, imageURL!).response() { (request, response, data, error) in
            if let error = error {
                print(error.localizedDescription)
                completion(nil, error)
            } else {
                print("image response")
                let photo = PTWPhoto()
                photo.id = id
                photo.sourceURL = source
                photo.imageData = data
                photo.createdTime = photo.createdTimeAsDate(created_time!)
                completion(photo, nil)
            }
        }
    }

}