Can not verify encrypted private key

Question

I have the context below:

I use openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -sha256 -days 365 -nodes to generate key and cert files.

Then I use openssl pkcs8 -in key.pem -out key-encrypted.pem -topk8 to encrypt my private key.

The code works with key.pem and cert.pem, but has a problem with key-encrypted.pem and cert.pem.

When it goes to this line to generate private key, it shows an error that java.security.InvalidKeyException: IOException : DerValue.getBigIntegerInternal, not expected 48.

I load my cert file and create the X509Certificate object as below:

byte[] certData = Files.readAllBytes(certPath);
InputStream st = new ByteArrayInputStream(certData)
CertificateFactory certFactory = CertificateFactory.getInstance("X.509");
certificate = (X509Certificate) certFactory.generateCertificate(st);

Then I load my private key file as below:

List<String> keys = Files.readAllLines(keyPath);
//I have removed the first and last line of the key content:
//-----BEGIN PRIVATE KEY----- and -----END PRIVATE KEY----- lines
String key = String.join("", keys);

//Parse it into PKCS8
byte[] decoded = Base64.decodeBase64(key);
KeyFactory kf = KeyFactory.getInstance("RSA");
PKCS8EncodedKeySpec ks = new PKCS8EncodedKeySpec(decoded);

//Now I use kf to generate private key
PrivateKey pk = kf.generatePrivate(ks);

//Then I use the following code to verify
Signature s = Signature.getInstance("SHA256withRSA");
s.initSign(pk);

byte[] sign = s.sign();
s.initVerify(certificate.getPublicKey());
s.verify(sign);

Answer 1

Newer openssl-pkcs8 versions use PKCS#5 v2.0/PBES2 by default, more specifically aes256 with PBKDF2 and hmacWithSHA256 (as of v1.1.0 in 2016, s. comment).
While EncryptedPrivateKeyInfo suggested in the other answer supports PBES2 algorithms in Java versions as of 19 (tested for 19-21) this is not the case for older Java versions such as Java 8 or even Java 18 and results in exceptions (which vary depending on the Java version, e.g. IOException: ObjectIdentifier() -- data isn't an object ID (tag = 48) with Java 8 or NoSuchAlgorithmException: PBES2 SecretKeyFactory not available with Java 18).

In the case of a Java version for which EncryptedPrivateKeyInfo does not support PBES2 algorithms org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo is an option, which also supports PBES2 algorithms:

import org.bouncycastle.asn1.pkcs.PrivateKeyInfo;
import org.bouncycastle.jce.provider.BouncyCastleProvider;
import org.bouncycastle.openssl.jcajce.JcaPEMKeyConverter;
import org.bouncycastle.openssl.jcajce.JceOpenSSLPKCS8DecryptorProviderBuilder;
import org.bouncycastle.operator.InputDecryptorProvider;
import org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo;
...
Security.addProvider(new BouncyCastleProvider());
// Load DER encoded encrypted key
byte[] encryptedPkcs8Der = Base64.decodeBase64(key);
PKCS8EncryptedPrivateKeyInfo pkcs8EncryptedPrivateKeyInfo = new PKCS8EncryptedPrivateKeyInfo(encryptedPkcs8Der);
// Decrypt encrypted key
JcaPEMKeyConverter jcaPEMKeyConverter = new JcaPEMKeyConverter();
InputDecryptorProvider inputDecryptorProvider = new JceOpenSSLPKCS8DecryptorProviderBuilder().build("<your password>".toCharArray());
PrivateKeyInfo privateKeyInfo = pkcs8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(inputDecryptorProvider);
// Import decrypted key
PrivateKey privateKey = jcaPEMKeyConverter.getPrivateKey(privateKeyInfo);
...
Signature s = Signature.getInstance("SHA256withRSA");
s.initSign(privateKey);
...