PBKDF2 with bouncycastle in Java

I'm trying to securely store a password in a database and for that I chose to store its hash generated using the PBKDF2 function. I want to do this using the bouncy castle library but I don't know why I cannot get it to work by using the JCE interface... The problem is that generating the hash in 3 different modes:
1. using the PBKDF2WithHmacSHA1 secret key factory provided by sun
2. using the bouncy castle api directly
3. using the bouncy castle through JCE
results in 2 distinct values: one common to the first two and one for the third.

Here is my code:

    //Mode 1

    SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1");
    KeySpec keyspec = new PBEKeySpec("password".toCharArray(), salt, 1000, 128);
    Key key = factory.generateSecret(keyspec);
    System.out.println(key.getClass().getName());
    System.out.println(Arrays.toString(key.getEncoded()));

    //Mode 2

    PBEParametersGenerator generator = new PKCS5S2ParametersGenerator();
    generator.init(PBEParametersGenerator.PKCS5PasswordToUTF8Bytes(("password").toCharArray()), salt, 1000);
    KeyParameter params = (KeyParameter)generator.generateDerivedParameters(128);
    System.out.println(Arrays.toString(params.getKey()));

    //Mode 3

    SecretKeyFactory factorybc = SecretKeyFactory.getInstance("PBEWITHHMACSHA1", "BC");
    KeySpec keyspecbc = new PBEKeySpec("password".toCharArray(), salt, 1000, 128);
    Key keybc = factorybc.generateSecret(keyspecbc);
    System.out.println(keybc.getClass().getName());
    System.out.println(Arrays.toString(keybc.getEncoded()));
    System.out.println(keybc.getAlgorithm());

I know that PBKDF2 is implemented using HMAC SHA1 so that is why i chose as algorithm in the last method the "PBEWITHHMACSHA1" which i took from the bouncy castle java docs.

The output is the following:

com.sun.crypto.provider.SunJCE_ae
[-53, 29, 113, -110, -25, 76, 115, -127, -64, 74, -63, 102, 75, 81, -21, 74]
[-53, 29, 113, -110, -25, 76, 115, -127, -64, 74, -63, 102, 75, 81, -21, 74]
org.bouncycastle.jce.provider.JCEPBEKey
[14, -47, -87, -16, -117, -31, 91, -121, 90, -68, -82, -31, -27, 5, -93, -67, 30, -34, -64, -40]
PBEwithHmacSHA

Any ideas?

Solution

In short, the reason for the difference is that PBKDF2 algorithm in modes #1 and #2 uses PKCS #5 v2 scheme 2 (PKCS5S2) for iterative key generation, but the BouncyCastle provider for "PBEWITHHMACSHA1" in mode #3 uses the PKCS #12 v1 (PKCS12) algorithm instead. These are completely different key-generation algorithms, so you get different results.

More detail on why this is so and why you get different sized results is explained below.

First, when you're constructing a JCE KeySpec, the keyLength parameter only expresses "a preference" to the provider what key size you want. From the API docs:

Note: this is used to indicate the preference on key length for variable-key-size ciphers. The actual key size depends on each provider's implementation.

The Bouncy Castle providers don't appear to respect this parameter, judging from the source of JCEPBEKey, so you should expect to get a 160-bit key back from any BC provider which uses SHA-1 when using the JCE API.

You can confirm this by programmatically accessing the getKeySize() method on the returned keybc variable in your test code:

Key keybc = factorybc.generateSecret(keyspecbc);
// ...
Method getKeySize = JCEPBEKey.class.getDeclaredMethod("getKeySize");
getKeySize.setAccessible(true);
System.out.println(getKeySize.invoke(keybc)); // prints '160'

Now, to understand what the "PBEWITHHMACSHA1" provider corresponds to, you can find the following in the source for BouncyCastleProvider:

put("SecretKeyFactory.PBEWITHHMACSHA1", 
    "org.bouncycastle.jce.provider.JCESecretKeyFactory$PBEWithSHA");

And the implementation of JCESecretKeyFactory.PBEWithSHA looks like this:

public static class PBEWithSHA
    extends PBEKeyFactory
{
    public PBEWithSHA()
    {
        super("PBEwithHmacSHA", null, false, PKCS12, SHA1, 160, 0);
    }
}

You can see above that this key factory uses the PKCS #12 v1 (PKCS12) algorithm for iterative key generation. But the PBKDF2 algorithm that you want to use for password hashing uses PKCS #5 v2 scheme 2 (PKCS5S2) instead. This is why you're getting different results.

I had a quick look through the JCE providers registered in BouncyCastleProvider, but couldn't see any key generation algorithms that used PKCS5S2 at all, let alone one which also uses it with HMAC-SHA-1.

So I guess you're stuck with either using the Sun implementation (mode #1 above) and losing portability on other JVMs, or using the Bouncy Castle classes directly (mode #2 above) and requiring the BC library at runtime.

Either way, you should probably switch to 160-bit keys, so you aren't truncating the generated SHA-1 hash unnecessarily.