Search code examples
c#encryptionutf-8aesrncryptor

AES decrypting non text files end up in corrupt data


I'm writing a Windows app in C# which has to interact with a Mac app (written in Cocoa). This app encrypts files in AES with CBC (IV, a key, salt, HMAC). I don't know a lot about encryption but I think that's what it does. The Cocoa library we use is RNCryptor. They have a C# version which is what I'm using on the Windows side (with a few modifications, mainly to use byte[] instead of Strings).

Now text files are decrypted correctly, but other files (for example, a PNG file), end up corrupted (the correct file on the right, and the corrupted on the left, using UTF8 encoding, you can see there's a lot of diamonds with ?s):

differences

I believe this is due to the encoding of the file, but I tried UTF8, Default, Unicode, ASCII... and the output files are always corrupted with different file sizes, being ASCII and the default encoding (UTF16 I believe) the closest in size.

This is the RNCryptor modified code I used:

public byte[] Decrypt (byte[] encryptedBase64, string password)
    {
        PayloadComponents components = this.unpackEncryptedBase64Data (encryptedBase64);

        if (!this.hmacIsValid (components, password)) {
            return null;
        }

        byte[] key = this.generateKey (components.salt, password);

        byte[] plaintextBytes = new byte[0];

        switch (this.aesMode) {
            case AesMode.CTR:
                // Yes, we are "encrypting" here.  CTR uses the same code in both directions.
                plaintextBytes = this.encryptAesCtrLittleEndianNoPadding(components.ciphertext, key, components.iv);
                break;

            case AesMode.CBC:
                plaintextBytes = this.decryptAesCbcPkcs7(components.ciphertext, key, components.iv);
                break;
        }

        return plaintextBytes;
    }

    private byte[] decryptAesCbcPkcs7 (byte[] encrypted, byte[] key, byte[] iv)
    {
        var aes = Aes.Create();
        aes.Mode = CipherMode.CBC;
        aes.Padding = PaddingMode.PKCS7;
        var decryptor = aes.CreateDecryptor(key, iv);

        string plaintext;
        using (MemoryStream msDecrypt = new MemoryStream(encrypted))
        {
            using (CryptoStream csDecrypt = new CryptoStream(msDecrypt, decryptor, CryptoStreamMode.Read))
            {
                using (StreamReader srDecrypt = new StreamReader(csDecrypt))
                {
                    plaintext = srDecrypt.ReadToEnd();
                }
            }
        }

        return Encoding.UTF8.GetBytes(plaintext);
    }

    private PayloadComponents unpackEncryptedBase64Data (byte[] encryptedBase64)
    {
        List<byte> binaryBytes = new List<byte>();
        binaryBytes.AddRange (encryptedBase64);

        PayloadComponents components;
        int offset = 0;

        components.schema = binaryBytes.GetRange(0, 1).ToArray();
        offset++;

        this.configureSettings ((Schema)binaryBytes [0]);

        components.options = binaryBytes.GetRange (1, 1).ToArray();
        offset++;

        components.salt = binaryBytes.GetRange (offset, Cryptor.saltLength).ToArray();
        offset += components.salt.Length;

        components.hmacSalt = binaryBytes.GetRange(offset, Cryptor.saltLength).ToArray();
        offset += components.hmacSalt.Length;

        components.iv = binaryBytes.GetRange(offset, Cryptor.ivLength).ToArray();
        offset += components.iv.Length;

        components.headerLength = offset;

        components.ciphertext = binaryBytes.GetRange (offset, binaryBytes.Count - Cryptor.hmac_length - components.headerLength).ToArray();
        offset += components.ciphertext.Length;

        components.hmac = binaryBytes.GetRange (offset, Cryptor.hmac_length).ToArray();

        return components;

    }

    private bool hmacIsValid (PayloadComponents components, string password)
    {
        byte[] generatedHmac = this.generateHmac (components, password);

        if (generatedHmac.Length != components.hmac.Length) {
            return false;
        }

        for (int i = 0; i < components.hmac.Length; i++) {
            if (generatedHmac[i] != components.hmac[i]) {
                return false;
            }
        }
        return true;
    }

And this is my code decrypting and writing the file:

 byte[] decryptedFile = this.decryptor.Decrypt(File.ReadAllBytes(filePath), password);
 File.WriteAllBytes(filePath, decryptedFile);

What can be wrong here? Thanks in advance.


Solution

  • The problem is in your use of StreamReader when decrypting. StreamReader reads text (UTF-8 here), not arbitrary binary data. One solution would be to read the data into a MemoryStream, and use its ToArray() method to get the resulting byte[].