Syncing Core Data with Database - Updating Records is Very Slow

Question

The app I am working on has to stay in sync with a database through web service calls.

You initially have to insert around 300k records, which takes a good bit of time. After the initial download, it calls an update web service to get all of the records that have since been changed.

During the update call, I have to first see if that object exists already, which really slows down that update process. Usually in the morning there are around 5k records to update, and that takes almost as long as it does to simply insert 300k records.

Code is below. Is there a faster, more efficient way to do this?

code in loop when iterating through JSON records from server

    if(isUpdating)
    {
        CaseStatusCode *oldObj = [CaseStatusCode doesExist:[record valueForKey:@"JSONData"] uid:[record valueForKey:@"RecordUID"] inContext:temporaryContext];
        if(oldObj) //it exists
        {
            [oldObj initWithJSONSting:[record valueForKey:@"JSONData"]];
        }
        else //create new
        {
            NSEntityDescription *desc = [NSEntityDescription entityForName:NSStringFromClass([CaseStatusCode class]) inManagedObjectContext:temporaryContext];
            CaseStatusCode *worker = [[CaseStatusCode alloc] initWithEntity:desc insertIntoManagedObjectContext:temporaryContext];
            [worker initWithJSONSting:[record valueForKey:@"JSONData"]];
        }
    }

doesExist function

+ (CaseStatusCode *)doesExist:(NSString *)jsonString uid:(NSString *)uid inContext:(NSManagedObjectContext *)moc
{
    SBJsonParser *parser = [[SBJsonParser alloc] init];
    NSError *e;
    NSDictionary *jsonObj =  [parser objectWithString:jsonString error:&e];

    if(uid || NULL_TO_NIL([jsonObj valueForKey:@"CaseStatusCodeUID"]))
    {
        if(jsonObj)
        {
            uid = [jsonObj valueForKey:@"CaseStatusCodeUID"];
        }
        NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:[self entityName]];
        [fetchRequest setPredicate:[NSPredicate predicateWithFormat:@"caseStatusCodeUID ==[cd] %@", uid]];
        [fetchRequest setFetchLimit:1];

        NSError *error = nil;

        if ([moc countForFetchRequest:fetchRequest error:&error] == 0) {
            NSLog(@"!!!WARN: NO Object Matches for pred: %@", fetchRequest.predicate);
            return nil;
        }
        else
        {
            // fetch your object
            CaseStatusCode *result = [[moc executeFetchRequest:fetchRequest error:&error] lastObject];
            if (error != nil) {
                NSLog(@"ERROR: %@ %@", [error localizedDescription], [error userInfo]);
                return nil;
            }
            return result;
        }
    }
    return nil;
}

init code

- (void)initWithJSONSting:(NSString *)jsonString
{
    SBJsonParser *parser = [[SBJsonParser alloc] init];
    NSError *e;
    NSDictionary *jsonObj =  [parser objectWithString:jsonString error:&e];

    if(e)
    {
        NSLog(@"JSON Error Creating State: %@", [e localizedDescription]);
    }
    else
    {
        MySingleton *globals = [MySingleton sharedInstance];
        self.token = globals.token;

        self.CaseStatusCodeUID = NULL_TO_NIL([jsonObj valueForKey:@"CaseStatusCodeUID"]);
        self.CaseStatusCode = NULL_TO_NIL([jsonObj valueForKey:@"CaseStatusCode"]);
        self.Active = NULL_TO_NIL([jsonObj valueForKey:@"Active"]);
        self.desc = NULL_TO_NIL([jsonObj valueForKey:@"Description"]);
        self.BaseStatus = NULL_TO_NIL([jsonObj valueForKey:@"BaseStatus"]);

    }
}

Thanks.

---

EDIT I am attempting to follow Apples example. I am building an NSArray of UIDs, then building a fetch request using an IN predicate. I get 0 results back, and I verified that there is a object of that type with that UID.

uidArray = (NSMutableArray *)[uidArray sortedArrayUsingSelector:@selector(compare:)];
viewName = @"CaseStatusCode";
uidKey = @"caseStatusCodeUID";

// create the fetch request to get all CaseStatusCode matching the IDs
NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init];
[fetchRequest setEntity:[NSEntityDescription entityForName:viewName inManagedObjectContext:temporaryContext]];
[fetchRequest setPredicate: [NSPredicate predicateWithFormat: @"%@ IN %@",uidKey, uidArray]];

// Make sure the results are sorted as well.
[fetchRequest setSortDescriptors:@[ [[NSSortDescriptor alloc] initWithKey:uidKey ascending:YES] ]];
// Execute the fetch.
NSError *error;
NSArray *matchingObjectsArray = [temporaryContext executeFetchRequest:fetchRequest error:&error];
NSLog(@"fetchRequest: %@", fetchRequest.description);
NSLog(@"# RECORDS TO UPDATE: %d",matchingObjectsArray.count );

Heres the resulting FetchRequest:

<NSFetchRequest: 0x1f5d69d0> (entity: CaseStatusCode; predicate: ("caseStatusCodeUID" IN {"ef236614-aaa2-49ae-8713-fd4847948498"}); sortDescriptors: ((
    "(caseStatusCodeUID, ascending, compare:)"
)); type: NSManagedObjectResultType; )

And like I said a caseStatusCode with that UID does exist. Also, if I don't set a predicate I do get all of my CaseStatusCodes so I know the fetch is working.. there is just something with the predicate.

EDIT #2

Ok I was being dumb above, the correct predicate format uses %K:

[fetchRequest setPredicate: [NSPredicate predicateWithFormat: @"%K IN %@",uidKey, uidArray]];

EDIT #3

What would be the best way to iterate through the idArray and the resultsArray? ie

You end up with two sorted arrays—one with the employee IDs passed into the fetch request, and one with the managed objects that matched them. To process them, you walk the sorted lists following these steps:

1. Get the next ID and Employee. If the ID doesn't match the Employee ID, create a new Employee for that ID.
2. Get the next Employee: if the IDs match, move to the next ID and Employee.

Answer 1

I suggest reading this Implementing Find-or-Create Efficiently sounds look this is in the area you're working with.