Decrypting message with RSA-OAEP and Android Keystore: IllegalBlockSizeException

Question

I have the following code inside a try block that should generate a RSA public/private keypair use the public key to encrypt a message and decrypt again with the private key:

KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance(
         KeyProperties.KEY_ALGORITHM_RSA, "AndroidKeyStore");
keyPairGenerator.initialize(new KeyGenParameterSpec.Builder(
                 "key1",
                 KeyProperties.PURPOSE_DECRYPT | KeyProperties.PURPOSE_ENCRYPT)
                 .setDigests(KeyProperties.DIGEST_SHA256, KeyProperties.DIGEST_SHA512)
                 .setEncryptionPaddings(KeyProperties.ENCRYPTION_PADDING_RSA_OAEP)
                 .build());
KeyPair keyPair = keyPairGenerator.generateKeyPair();
byte[] src = "hello world".getBytes();

Cipher cipher = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding");
cipher.init(Cipher.ENCRYPT_MODE, keyPair.getPublic());
byte[] cipherData = cipher.doFinal(src);

Cipher cipher2 = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding");
cipher2.init(Cipher.DECRYPT_MODE, keyPair.getPrivate());
byte[] msg = cipher2.doFinal(cipherData);

Taken mostly from here and here.

The final line throws an exception of type javax.crypto.IllegalBlockSizeException with no message/further details. The three lines in logcat before the exception are

 E keymaster1_device: Finish send cmd failed
 E keymaster1_device: ret: 0
 E keymaster1_device: resp->status: -1000

in case that matters at all.

Does anyone have an idea what could be going wrong?

Using minSdkVersion 23

Edit: I just realised, if I use PKCS#1 v1.5 padding it works. That helps me for now, but I'd still like to try get it work with OAEP.

Answer 1

You need to put into the cipher the algorithm parameter spec when you encrypt

if (algorithmParameterSpec != null) {
            encrypter.init(Cipher.ENCRYPT_MODE, getKey(), algorithmParameterSpec)
        }

algorithmParameterSpec is

OAEPParameterSpec("SHA-256",
                    "MGF1",
                    MGF1ParameterSpec.SHA256,
                    PSource.PSpecified.DEFAULT)