Unique IVs producing identical ciphertext using attr_encrypted

Question

Using the most basic setup:

class User < ActiveRecord::Base
  attr_encrypted :name, 
                 key: 'This is a key that is 256 bits!!', 
                 encode: true, 
                 encode_iv: true, 
                 encode_salt: true
end

The results look like this in the database when supplying an identical name:

╔════╦══════════════════════════════╦═══════════════════╗
║ id ║ encrypted_name               ║ encrypted_name_iv ║
╠════╬══════════════════════════════╬═══════════════════╣
║ 1  ║ aVXZb1b317nroumXVBdV9pGxA2o= ║ JyE7wHups+3upY5e  ║
║ 2  ║ aVXZb1b317nroumXVBdV9pGxA2o= ║ uz/ktrtbUAksg5Vp  ║
╚════╩══════════════════════════════╩═══════════════════╝

Why is the ciphertext identical? Isn't that the part of the point of iv, which the gem is using by default?

Answer 1

Update: the following is the original post explaining the whole problem, the issue is fixed now, see the bottom of this answer for a solution.

I am quite sure you noticed a rather nasty security issue in the encryptor gem (the gem that is used by attr_encrypted to do the actual encryptions).

The problem is that when using the aes-256-gcm algorithm (or any of the AES GCM algorithms), the initialization vector (IV) is currently indeed not taken into account when encrypting. The issue does not affect other algorithms but unfortunately the aes-256-gcm is the default algorithm in attr_encrypted.

As it turns out, it is the order of setting the IV vs. the encryption key what causes the issue. When IV is set before the key (as is in the gem), the IV is not taken into account but it is if set after the key.

Some tests to prove the problem:

While taking parts of the encryptor gem code, I created the simplest test case to prove the problem (tested under ruby 2.3.0 compiled against OpenSSL version "1.0.1f 6 Jan 2014"):

def base64_enc(bytes)
  [bytes].pack("m")
end

def test_aes_encr(n, cipher, data, key, iv, iv_before_key = true)
  cipher = OpenSSL::Cipher.new(cipher)
  cipher.encrypt

  # THIS IS THE KEY PART OF THE ISSUE
  if iv_before_key
    # this is how it's currently present in the encryptor gem code
    cipher.iv = iv
    cipher.key = key
  else
    # this is the version that actually works
    cipher.key = key
    cipher.iv = iv
  end

  if cipher.name.downcase.end_with?("gcm")
    cipher.auth_data = ""
  end

  result = cipher.update(data)
  result << cipher.final

  puts "#{n} #{cipher.name}, iv #{iv_before_key ? "BEFORE" : "AFTER "} key: " +
           "iv=#{iv}, result=#{base64_enc(result)}"
end

def test_encryption
  data = "something private"
  key = "This is a key that is 256 bits!!"

  # control tests using AES-256-CBC
  test_aes_encr(1, "aes-256-cbc", data, key, "aaaabbbbccccdddd", true)
  test_aes_encr(2, "aes-256-cbc", data, key, "eeeeffffgggghhhh", true)
  test_aes_encr(3, "aes-256-cbc", data, key, "aaaabbbbccccdddd", false)
  test_aes_encr(4, "aes-256-cbc", data, key, "eeeeffffgggghhhh", false)

  # failing tests using AES-256-GCM
  test_aes_encr(5, "aes-256-gcm", data, key, "aaaabbbbcccc", true)
  test_aes_encr(6, "aes-256-gcm", data, key, "eeeeffffgggg", true)
  test_aes_encr(7, "aes-256-gcm", data, key, "aaaabbbbcccc", false)
  test_aes_encr(8, "aes-256-gcm", data, key, "eeeeffffgggg", false)
end

Running test_encryption which encrypts a text using AES-256-CBC and then using AES-256-GCM, each time with two different IVs in two regimes (IV set before/after key), gets us the following results:

# control tests with CBC:
1 AES-256-CBC, iv BEFORE key: iv=aaaabbbbccccdddd, result=4IAGcazRmEUIRDE3ZpEgoS0Nmm1/+nrd5VT2/Xab0WM=
2 AES-256-CBC, iv BEFORE key: iv=eeeeffffgggghhhh, result=T7um2Wgb2vw1r4uryF3xnBeq+KozuetjKGItfNKurGE=
3 AES-256-CBC, iv AFTER  key: iv=aaaabbbbccccdddd, result=4IAGcazRmEUIRDE3ZpEgoS0Nmm1/+nrd5VT2/Xab0WM=
4 AES-256-CBC, iv AFTER  key: iv=eeeeffffgggghhhh, result=T7um2Wgb2vw1r4uryF3xnBeq+KozuetjKGItfNKurGE=

# the problematic tests with GCM:
5 id-aes256-GCM, iv BEFORE key: iv=aaaabbbbcccc, result=Tl/HfkWpwoByeYRz6Mz4yIo=
6 id-aes256-GCM, iv BEFORE key: iv=eeeeffffgggg, result=Tl/HfkWpwoByeYRz6Mz4yIo=
7 id-aes256-GCM, iv AFTER  key: iv=aaaabbbbcccc, result=+4Iyn7RSDKimTQi0S3gn58E=
8 id-aes256-GCM, iv AFTER  key: iv=eeeeffffgggg, result=3m9uEDyb9eh1RD3CuOCmc50=

These tests show that while the order of setting IV vs. key is not relevant for CBC, it is for GCM. More importantly, the encrypted result in CBC is different for two different IVs, whereas it is not for GCM if IV set before the key.

I just created a pull request to fix this issue in the encryptor gem. Practically, you have a few options now:

• Wait till a new version of the encryptor gem is released.

• Use also salt with attr_encrypted. You should use salt anyway to further secure the encrypted data.

The very unfortunate thing is that all already encrypted data will become undecipherable after the fix as suddenly the IVs will be taken into account.

Update: encryptor 3.0.0 available

You can now upgrade the encryptor gem to version 3.0 in which the bug is fixed. Now, if this is the first time you use the encryptor or attr_encrypted gems you are all set and everything should work correctly.

If you have data that is already encrypted using encryptor 2.0.0, then you must manually re-encrypt the data after the gem upgrade, otherwise it will fail to decrypt correctly! You will be warned about this during the gem upgrade. The schematic procedure is as follows:

• You have to decrypt all your encrypted data using the Encryptor class (see the README for examples), using the :v2_gcm_iv => true option. This should correctly decrypt your data.
• Then you must encrypt the same data back again, now without this option (i.e. :v2_gcm_iv => false) but including the proper IV from your database.
• If you have production data, you will need to do this upgrade offline and immediately after the gem update to ensure no data corruption.

Update 2: issue in the openssl gem confirmed and fixed

FYI, it was recently confirmed that this had actually been an issue in the underlying ruby-openssl library and the bug has been fixed now. So, in the future, it is possible that even attr_encrypted gem version 2.x will actually work correctly when used with the new openssl-2.0.0 gem version (which is now in beta as of Sep 2016).