Search code examples
c++openssldiffie-hellman

Unable to derive Diffie Hellman shared secret with OpenSSL in C++


I'm trying to familiarise myself with the OpenSSL Diffie Hellman features and, in doing so, I've tried to create a simple programme which will generate two sets of Diffie Hellman private and public keys and then derive the shared secret. I have followed the Diffie Hellman tutorial on the OpenSSL wiki and I am able to generate the keys, however I am unable to derive the shared secret. My C++ (Linux) code is as follows:

#include <iostream>
#include <openssl/dh.h>
#include <openssl/engine.h>
#include <openssl/err.h>
#include <openssl/evp.h>
#include <openssl/rsa.h>

int main(int argc, char** argv)
{

    EVP_PKEY* params;

    EVP_PKEY_CTX *kctx1, *kctx2, *dctx1, *dctx2;

    unsigned char *skey1, *skey2;
    size_t skeylen1, skeylen2;
    EVP_PKEY *dh_1, *dh_2;
    BIO* bio_out = NULL;
    int result = 0;
    ENGINE* eng;

    BIO* fp = BIO_new_fp(stdout, BIO_NOCLOSE);

    // Initalise Diffie Hellman PKEY for client 1
    if(NULL == (dh_1 = EVP_PKEY_new())) {
        std::cout << "error 1" << std::endl;
    }

    // Initalise Diffie Hellman PKEY for client 2
    if(NULL == (dh_2 = EVP_PKEY_new())) {
        std::cout << "error 2" << std::endl;
    }

    // Initalise Diffie Hellman parameter PKEY
    if(NULL == (params = EVP_PKEY_new())) {
        std::cout << "error 3" << std::endl;
    }

    // Set Diffie Hellman paramerers
    if(1 != EVP_PKEY_set1_DH(params, DH_get_2048_256())) {
        std::cout << "error 4" << std::endl;
    }

    // Initalise client 1 PKEY Context
    if(!(kctx1 = EVP_PKEY_CTX_new(params, NULL))) {
        std::cout << "error 5" << std::endl;
    }

    // Initalise client 2 PKEY Context
    if(!(kctx2 = EVP_PKEY_CTX_new(params, NULL))) {
        std::cout << "error 6" << std::endl;
    }

    if(!kctx1) {
        std::cout << "error 7" << std::endl;
    }

    if(!kctx2) {
        std::cout << "error 8" << std::endl;
    }

    // Initalise both contexts key generators
    if(1 != EVP_PKEY_keygen_init(kctx1)) {
        std::cout << "error 9" << std::endl;
    }

    if(1 != EVP_PKEY_keygen_init(kctx2)) {
        std::cout << "error 10" << std::endl;
    }

    // Generate DH public and private keys for client 1
    if(1 != EVP_PKEY_keygen(kctx1, &dh_1)) {
        std::cout << "error 11" << std::endl;
    }

    // Generate DH public and private keys for client 2
    if(1 != EVP_PKEY_keygen(kctx2, &dh_2)) {
        std::cout << "error 12" << std::endl;
    }

    // EVP_PKEY_print_public(fp, dh_1, 3, NULL);

    // EVP_PKEY_print_public(fp, dh_2, 3, NULL);

    // Create key derivation context
    if(NULL == (dctx1 = EVP_PKEY_CTX_new(dh_1, NULL))) {
        std::cout << "error 13" << std::endl;
    }

    if(!dctx1) {
        std::cout << "error 14" << std::endl;
    }

    // Initalise first key derivation context
    if(1 != EVP_PKEY_derive_init(dctx1)) {
        std::cout << "error 15" << std::endl;
    }

    if(1 != EVP_PKEY_check(dctx1)) {
        std::cout << "error 16" << std::endl;
    }

    if(1 != EVP_PKEY_param_check(dctx1)) {
        std::cout << "error 17" << std::endl;
    }

    // Set first key derivation context peer key to the second DH PKEY
    if(1 != EVP_PKEY_derive_set_peer(dctx1, dh_2)) {
        std::cout << "error 18" << std::endl;
    }

    if(1 != EVP_PKEY_public_check(dctx1)) {
        std::cout << "error 19" << std::endl;
    }

    /* Determine buffer length */
    if(EVP_PKEY_derive(dctx1, NULL, &skeylen1) <= 0) {
    }

    // Assign memory for shared key variable
    skey1 = (unsigned char*)OPENSSL_malloc(skeylen1);

    if(result = EVP_PKEY_derive(dctx1, skey1, &skeylen1) <= 0) {
        std::cout << "Key: " << skey1 << std::endl;

        for(int i = 0; i < skeylen1; i++) {
            std::cout << std::hex << (unsigned int)skey1[i];
        }
    }

    ERR_print_errors_fp(stdout);

    std::cout << result << std::endl;
}

My main concerns are:

  1. What do I need to do to derive the shared secret?
  2. Do I need a new context for each key pair and key derivation?

Thanks!

EDIT:

I get the following errors and no shared key:

140690271102784:error:05079079:Diffie-Hellman routines:DH_check_ex:unable to check generator:../crypto/dh/dh_check.c:92:

140690271102784:error:05079076:Diffie-Hellman routines:DH_check_ex:check p not safe prime:../crypto/dh/dh_check.c:96:

140690271102784:error:0507B07B:Diffie-Hellman routines:DH_check_pub_key_ex:check pubkey too large:../crypto/dh/dh_check.c:190:

140690271102784:error:0507B07A:Diffie-Hellman routines:DH_check_pub_key_ex:check pubkey invalid:../crypto/dh/dh_check.c:192:

when I use

    if(NULL == (dctx1 = EVP_PKEY_CTX_new(dh_1, NULL))) {
        std::cout << "error 13" << std::endl;
    }

and I get the following errors

error 16
error 19

Key: �ȭ��U

c0c8add7c4550000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

139939742943040:error:060BA096:digital envelope routines:EVP_PKEY_check:operation not supported for this keytype:../crypto/evp/pmeth_gn.c:188:
139939742943040:error:0507C07D:Diffie-Hellman routines:dh_pkey_public_check:missing pubkey:../crypto/dh/dh_ameth.c:517:
139939742943040:error:05066064:Diffie-Hellman routines:compute_key:no private value:../crypto/dh/dh_key.c:183:
1

when I change "EVP_PKEY_CTX_new(dh_1, NULL)" to "EVP_PKEY_CTX_new(params, NULL)", like so:

    if(NULL == (dctx1 = EVP_PKEY_CTX_new(params, NULL))) {
        std::cout << "error 13" << std::endl;
    }

The difference between these two variables is that "dh_1" stores a DH keypair, whereas "params" contains the DH parameters. It appears that the derivation context initalises to some extent with the latter, though it does not have a private and public key associated with it.


Solution

  • With the version of the code that you posted I got slightly different errors to you. I get the following output from your code (linked against OpenSSL 1.1.1):

    error 16
    error 17
    140673674348352:error:060BA096:digital envelope routines:EVP_PKEY_check:operation not supported for this keytype:crypto/evp/pmeth_gn.c:187:
    140673674348352:error:05079076:Diffie-Hellman routines:DH_check_ex:check p not safe prime:crypto/dh/dh_check.c:93:
    0
    

    The first "error 16" output is simply because OpenSSL does not currently support the EVP_PKEY_check() call for Diffie-Hellman keys. So this can be safely ignored.

    The second "error 17" output (which is associated with the "check p not safe prime" error in the error queue) is more of an issue. This is because Diffie-Hellman keys come in two different varieties. By default OpenSSL uses PKCS#3 Diffie-Hellman keys. However it also supports X9.42 Diffie-Hellman keys. The function DH_get_2048_256() gives you a built-in set of parameters for X9.42 keys. However the function EVP_PKEY_set1_DH() expects the provided DH object to be a PKCS#3 key.

    This actually looks like a bug in OpenSSL to me (EVP_PKEY_set1_DH() should really detect what type of key it is and do the right thing), so I have raised the following OpenSSL issue for this: https://github.com/openssl/openssl/issues/10592

    You can work around this by replacing the EVP_PKEY_set1_DH() call with one to EVP_PKEY_assign() instead and specify the type as EVP_PKEY_DHX as follows:

        if(1 != EVP_PKEY_assign(params, EVP_PKEY_DHX, DH_get_2048_256())) {
            std::cout << "error 4" << std::endl;
        }
    

    Alternatively you might just use PKCS#3 parameters instead.

    Finally, the reason you get no shared secret derived is because you are only printing it if the EVP_PKEY_derive() call fails! It is actually succeeding!

    I made the following changes to your code and it works for me:

    --- derive.cpp  2019-12-09 11:11:15.493349734 +0000
    +++ derive-new.cpp  2019-12-09 11:14:59.348715074 +0000
    @@ -37,7 +37,7 @@
         }
    
         // Set Diffie Hellman paramerers
    -    if(1 != EVP_PKEY_set1_DH(params, DH_get_2048_256())) {
    +    if(1 != EVP_PKEY_assign(params, EVP_PKEY_DHX, DH_get_2048_256())) {
             std::cout << "error 4" << std::endl;
         }
    
    @@ -96,9 +96,11 @@
             std::cout << "error 15" << std::endl;
         }
    
    +#if 0
         if(1 != EVP_PKEY_check(dctx1)) {
             std::cout << "error 16" << std::endl;
         }
    +#endif
    
         if(1 != EVP_PKEY_param_check(dctx1)) {
             std::cout << "error 17" << std::endl;
    @@ -120,7 +122,7 @@
         // Assign memory for shared key variable
         skey1 = (unsigned char*)OPENSSL_malloc(skeylen1);
    
    -    if(result = EVP_PKEY_derive(dctx1, skey1, &skeylen1) <= 0) {
    +    if((result = EVP_PKEY_derive(dctx1, skey1, &skeylen1)) > 0) {
             std::cout << "Key: " << skey1 << std::endl;
    
             for(int i = 0; i < skeylen1; i++) {