Differences in reading private key with libp11 and pkcs11-engine

Question

I'm trying to implement SSL client authentication in C++ using credentials stored on a smartcard.

In essence, this means using the openssl library to initialize an SSL context using a certificate and a private key, then using this context for all future https connections.

The libraries i've come across to help me with this are libp11 and its pkcs11-engine module, found here: https://github.com/OpenSC/libp11 .

A scenario in which my code actually works as intended with our webserver is where the certificate is listed and retrieved via libp11 and the private key is retrieved by id using the pkcs11 engine:

PKCS11_enumerate_certs(slot->token, &certs, &ncerts);

X509 *cert = certs[0].x509;

EVP_PKEY *pkey = ENGINE_load_private_key(pkcs11_eng, "pkey_id", NULL, NULL);

if (!SSL_CTX_use_certificate(context->ctx, cert )) {
    throw SSLError::getLastError();
}
if (!SSL_CTX_use_PrivateKey(context->ctx, pkey )) {
    throw SSLError::getLastError();
}
if (!SSL_CTX_check_private_key(context->ctx)) {
    throw SSLError::getLastError();
}

However, for consistency and to keep things simple, it is preferred to use libp11 for both of these retrievals as this would also eliminate the use of a whole other component (the pkcs11-engine).

The issue i'm having is that when retrieving pkey with:

PKCS11_KEY *key = PKCS11_find_key(&certs[0]);
EVP_PKEY pkey = PKCS11_get_private_key(key)

And initializing ssl, the checks pass, but the following error is thrown by the SSL_connect() function:

error:80009005:Vendor defined:PKCS11_rsa_encrypt:General Error

Basically the private key works when it's retrieved via engine but an error is thrown when using libp11, which is strange because looking through the code on github for the engine, i noticed the same p11 calls being made that i was using.

If anyone has any experience with this topic and might know what's going on here, it would help me enormously.

Answer 1

This was my bad. I was initializing the pkcs11 engine with our own dll implementing pkcs11 but using the opensc dll when initializing p11. I guess the lesson learned here is always look carefully before copy/pasting code