SHA1 MessageDigest and CommonCrypto updates getting different results

Question

I'm trying to recreate the implementation of PasswordDeriveBytes from VB.NET and so far I have implemented in iOS Objective C code its giving me different results from a Java implementation.

The reason why we try to recreate the implementation of PasswordDeriveBytes is because the client server side is using it to encrypt/decrypt data and from what I've searched, PasswordDeriveBytes uses PBKDF1, which is outdated. The Java implementation is returning the expected encrypted value and is being decrypted by the server successfully. However the value returned from the iOS ObjC implementation is incorrect.

Below is the Java implementation of the Constructor, taken from this answer: Encryption Diff Between Java and C#

public static class PasswordDeriveBytes{

private final MessageDigest hash;

private final byte[] firstToLastDigest;
private final byte[] outputBuffer;

private int position = 0;

public PasswordDeriveBytes(String password, byte[] salt, int iterations) {
    try {
        this.hash = MessageDigest.getInstance("SHA-1");

        this.hash.update(password.getBytes("UTF-8"));
        this.hash.update(salt);
        this.firstToLastDigest = this.hash.digest();
        // At this point, the Obj-C and Java values are the same
        // this.firstToLastDigest = b8fa3d36....

        for (int i = 1; i < iterations - 1; i++) {
            System.out.println( "  Iterate " + i);
            hash.update(firstToLastDigest);
            hash.digest(firstToLastDigest, 0, firstToLastDigest.length);
        }

        this.outputBuffer = hash.digest(firstToLastDigest);
        // However at this point, they become different
        // Java has outputBuffer = f498e100...
        // Obj-C has outputBuffer = <d7d5fa71...

    } catch (UnsupportedEncodingException|NoSuchAlgorithmException | DigestException e) {
        throw new IllegalStateException("SHA-1 digest should always be available", e);
    }
}

While below is the Objective C Code of the constructor, using this library: https://github.com/TakahikoKawasaki/nv-ios-digest

@implementation PasswordDeriveBytesObjC
{
    SHA1 *hash;
    Byte *firstToLastDigest;
    Byte *outputBuffer;

    int position;
}

- (instancetype)initWithPassword:(NSString *)password salt:(NSData *)salt iterations:(int)iterations
{
    self = [[[self class] alloc] init];
    if (self){
        hash = [[SHA1 alloc] init];

        const char* ASCIIpassword = [password cStringUsingEncoding:NSUTF8StringEncoding];
        NSData *passwordData = [NSData dataWithBytes:ASCIIpassword length:strlen(ASCIIpassword)];

        [hash updateWith:[passwordData bytes] length:(CC_LONG)[passwordData length]];
        [hash updateWith:[salt bytes] length:(CC_LONG)[salt length]];
        firstToLastDigest = [hash final];
        // At this point, the Obj-C and Java values are the same
        // firstToLastDigest = <b8fa3d36....

        for ( int i = 1; i < iterations - 1; i++ ){
            [hash updateWith:firstToLastDigest length:(CC_LONG)strlen(firstToLastDigest)];
        }

        [hash updateWith:firstToLastDigest length:(CC_LONG)strlen(firstToLastDigest)];
        outputBuffer = [hash final];  
        // However at this point, they become different
        // Java has outputBuffer = f498e100...
        // Obj-C has outputBuffer = <d7d5fa71...

    }
    return self;
}

As far as I have researched, the implementation of hash.digest(input) of java is the same as [hash updateWith:firstToLastDigest length:(CC_LONG)strlen(firstToLastDigest)]; outputBuffer = [hash final]; in Obj-C but I am getting different results.

At this point I have no other ideas as to why they get different values, so any kind of guidance or suggestions are welcome.

Answer 1

Answering my own question after 4 days of ripping my hair out.

It seems that CommonCrypto's CC_SHA1_Final() does not reset the CC_SHA1_CTX object context, despite apple documentation stating that it does, which causes the discrepancies in output.

From Apple Docs: https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man3/CC_SHA1_Final.3cc.html

CC_SHA1_Final() places the message digest in md, which must have space for CC_SHA1_DIGEST_LENGTH == 20 bytes of output, and erases the CC_SHA1_CTX.

I had to edit the nv-ios-digest library and manually reset the CC_SHA1_CTX object in each - (unsigned char *)final method as displayed below:

- (unsigned char *)final
{
    CC_SHA1_Final(_digest, &_context);  // <-- _context does not reset
    CC_SHA1_Init(&_context);  // <-- manually reset the CC_SHA1_CTX object

    _description = [NSString stringWithFormat:
                    @"%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x",
                    _digest[ 0], _digest[ 1], _digest[ 2], _digest[ 3],
                    _digest[ 4], _digest[ 5], _digest[ 6], _digest[ 7],
                    _digest[ 8], _digest[ 9], _digest[10], _digest[11],
                    _digest[12], _digest[13], _digest[14], _digest[15],
                    _digest[16], _digest[17], _digest[18], _digest[19]];

    return _digest;
}

Hope this helps someone. :)