Search code examples
javaencryptionaeshmachmacsha1

Confused how a client and a server using AES and Hmac will verify each other


The scenario is this: I have a client and server talking. Here is the standard idea:

  • Use Diffie-Hellman to generate a secret key between the client and server.
  • Use this secret key for AES/CTR/PKCS7Padding cipher on both client and server.
  • Use Hmac on the original message
  • Encrypt the Hmac message with AES cipher

So this would allow the client and server to talk securely.

The relevant code example I'm looking at is the tutorial here: Tampered message with HMac, encryption with AES in CTR mode : Advanced Encryption Standard « Security « Java Tutorial

I'm able to generate the secret key for both the client and server. I'm able to encrypt this using Hmac and AES. Because encryption and decryption happen independently, I am unsure how to retrieve the relevant information required for decryption.

Here is the section I'm confused on:

cipher.init(Cipher.DECRYPT_MODE, key, ivSpec);

byte[] plainText = cipher.doFinal(cipherText, 0, ctLength);
int messageLength = plainText.length - hMac.getMacLength();

hMac.init(hMacKey);
hMac.update(plainText, 0, messageLength);

byte[] messageHash = new byte[hMac.getMacLength()];

If the client sends an encrypted message, how does the server retrieve the ivSpec, hMac.getMacLength(), and hMacKey? These items are required on the server to decrypt the message from the client.

I understand that the Initialization Vector (IV) can be retained from the ciphertext since it's appended to the beginning of the resulting ciphertext (I think I'll have to add it in manually since I don't think AES cipher does that?). However, the hMacKey and hMac length used to verify the message integrity remains a mystery.

On a last note, can someone explain what the purpose of this line is? Is this for encryption or decryption?

     cipherText[9] ^= '0' ^ '9';`

Solution

  • First, for this kind of thing use SSL if your doing it for real. For educational purposes, this stuff is cool.

    As others have said, this is an example that proves HMAC works when the cipher text is tampered with. Hence the cipherText[9] ^= '0' ^ '9';

    In order to use HMAC, you have to verify that the message authentication tag (whats returned by hmac) you got with your cipher text matches the one you are supposed to have. To do this in this code you: 1) decrypt the message using the key you negotiated, 2) compute the hmac of that text using a different key you negotiated and then 3) compare that the two are the same.

    Since you know those keys, you can decrypt the message and compute the mac. The mac length b the way, is a fixed property of the hmac function and is publicly known.

    Because you must have the key to generate the valid tag, if they match, then the message is authentic.

    As a note, this code is likely insecure since you need to MAC the ciphertext, not plaintext. If you don't, you end up with problems like the Padding oracle attack that broke a bunch of sites secure cookie implementations. Use SSL for these kind of things.