I am trying to encode ECDSA Curve448 key into a small and portable byte array. I am trying to use DER to accomplish this, as the low-level APIs do not work with Curve448. However, when I use the following code:
std::vector<std::vector<uint8_t>> vecs;
uint8_t* buf = nullptr;
EVP_PKEY_CTX *pctx = EVP_PKEY_CTX_new_id(EVP_PKEY_ED448, NULL);
EVP_PKEY_keygen_init(pctx);
EVP_PKEY* pkey;
EVP_PKEY_keygen(pctx, &pkey);
EVP_PKEY_CTX_free(pctx);
size_t n = i2d_PublicKey(pkey, &buf);
ERR_print_errors_fp(stderr);
vecs.emplace_back(buf, buf + n);
n is set to -1, and so vecs.emplace_back fails. ERR_print_errors_fp prints:
140691149055104:error:0D0A40A7:asn1 encoding routines:i2d_PublicKey:unsupported public key type:../crypto/asn1/i2d_pu.c:35
How do I solve this problem, or is there a better way I should go about this?
I am using openssl v1.1.1 from debian experimental.
Are you trying to write out a private key or a public key? I ask because your code above has size_t n = i2d_PrivateKey(pkey, &buf);, i.e. attempting to encode a private key, but the error message says i2d_PublicKey:unsupported public key type.
Anyway i2d_PublicKey is for writing out public keys in old-style "traditional" format. Since ED448 is a new algorithm, there is no such format defined for it. Instead you must use SubjectPublicKeyInfo format. The OpenSSL function for that is the (confusingly similar) i2d_PUBKEY(). The man page is here: