Dispatch queues and asynchronous RNCryptor

Question

This is a follow-up to Asynchronously decrypt a large file with RNCryptor on iOS

I've managed to asynchronously decrypt a large, downloaded file (60Mb) with the method described in this post, corrected by Calman in his answer.

It basically goes like this:

int blockSize = 32 * 1024;
NSInputStream *cryptedStream = [NSInputStream inputStreamWithFileAtPath:...];
NSOutputStream *decryptedStream = [NSOutputStream output...];

[cryptedStream open];
[decryptedStream open];

RNDecryptor *decryptor = [[RNDecryptor alloc] initWithPassword:@"blah" handler:^(RNCryptor *cryptor, NSData *data) {
    NSLog("Decryptor recevied %d bytes", data.length);
    [decryptedStream write:data.bytes maxLength:data.length];
    if (cryptor.isFinished) {
        [decryptedStream close];
        // call my delegate that I'm finished with decrypting
    }
}];

while (cryptedStream.hasBytesAvailable) {
    uint8_t buf[blockSize];
    NSUInteger bytesRead = [cryptedStream read:buf maxLength:blockSize];
    NSData *data = [NSData dataWithBytes:buf length:bytesRead];

    [decryptor addData:data];
    NSLog("Sent %d bytes to decryptor", bytesRead);
}

[cryptedStream close];
[decryptor finish];

However, I'm still facing a problem: the whole data is loaded in memory before being decrypted. I can see a bunch of "Sent X bytes to decryptor", and after that, the same bunch of "Decryptor recevied X bytes" in the console, when I'd like to see "Sent, received, sent, receives, ...".

That's fine for small (2Mb) files, or with large (60Mb) files on simulator; but on a real iPad1 it crashes due to memory constraints, so obviously I can't keep this procedure for my production app.

I feel like I need to send the data to the decryptor by using dispatch_async instead of blindly sending it in the while loop, however I'm completely lost. I've tried:

• creating my own queue before the while, and using dispatch_async(myQueue, ^{ [decryptor addData:data]; });
• the same, but dispatching the whole code inside of the while loop
• the same, but dispatching the whole while loop
• using RNCryptor-provided responseQueue instead of my own queue

Nothing works amongst these 4 variants.

I don't have a complete understanding of dispatch queues yet; I feel the problem lies here. I'd be glad if somebody could shed some light on this.

Answer 1

If you only want to process one block at a time, then only process a block when the first block calls you back. You don't need a semaphore to do that, you just need to perform the next read inside the callback. You might want an @autoreleasepool block inside of readStreamBlock, but I don't think you need it.

When I have some time, I'll probably wrap this directly into RNCryptor. I opened Issue#47 for it. I am open to pull requests.

// Make sure that this number is larger than the header + 1 block.
// 33+16 bytes = 49 bytes. So it shouldn't be a problem.
int blockSize = 32 * 1024;

NSInputStream *cryptedStream = [NSInputStream inputStreamWithFileAtPath:@"C++ Spec.pdf"];
NSOutputStream *decryptedStream = [NSOutputStream outputStreamToFileAtPath:@"/tmp/C++.crypt" append:NO];

[cryptedStream open];
[decryptedStream open];

// We don't need to keep making new NSData objects. We can just use one repeatedly.
__block NSMutableData *data = [NSMutableData dataWithLength:blockSize];
__block RNEncryptor *decryptor = nil;

dispatch_block_t readStreamBlock = ^{
  [data setLength:blockSize];
  NSInteger bytesRead = [cryptedStream read:[data mutableBytes] maxLength:blockSize];
  if (bytesRead < 0) {
    // Throw an error
  }
  else if (bytesRead == 0) {
    [decryptor finish];
  }
  else {
    [data setLength:bytesRead];
    [decryptor addData:data];
    NSLog(@"Sent %ld bytes to decryptor", (unsigned long)bytesRead);
  }
};

decryptor = [[RNEncryptor alloc] initWithSettings:kRNCryptorAES256Settings
                                         password:@"blah"
                                          handler:^(RNCryptor *cryptor, NSData *data) {
                                            NSLog(@"Decryptor recevied %ld bytes", (unsigned long)data.length);
                                            [decryptedStream write:data.bytes maxLength:data.length];
                                            if (cryptor.isFinished) {
                                              [decryptedStream close];
                                              // call my delegate that I'm finished with decrypting
                                            }
                                            else {
                                              // Might want to put this in a dispatch_async(), but I don't think you need it.
                                              readStreamBlock();
                                            }
                                          }];

// Read the first block to kick things off    
readStreamBlock();