How to encrypt a string using OpenSSL C library and a public key file?

What is the recommended way of encrypting a short std::string into another std::string using the openssl C library (not the command-line tool of the same name) using a public keyfile, and knowing the algorithm? (in this case the string is no larger than ~100 bytes, keyfile is in .pem format, algorithm can be any asymmetric one like RSA/ECDSA/etc).

I am looking to write a function with an interface like so: bool EncryptString(const std::string& InStr, const std::string& InPublicKey, std::string& OutString).

Looking through documentation it seems like EVP_PKEY_encrypt is the function to use, which requires an EVP_PKEY_CTX *ctx variable. My assumption is, this variable should be initialized with EVP_PKEY_CTX_new, which in turn requires an EVP_PKEY *pkey and an ENGINE *e. Those I don't know how to initialize, and searching through the documentation leaves me very confused.

Maybe these functions are not the easiest approach, I am not familiar with this library at all and have no cryptography knowledge. I simply care about a black-box way of converting a string to an encrypted string.

Thank you

Solution

As it turned out in the comments, RSA is an acceptable option for you.

When implementing RSA with OpenSSL, the following steps are required for encryption:

Loading the public key
Creating and initializing the context
Specifying the padding
Encryption

The implementation below for encryption with RSA and OpenSSL runs successfully on my machine and shows how the functions are called (without exception handling for simplicity):

#include <openssl/pem.h>
#include <string>
...

bool EncryptString(const std::string& InStr /*plaintext*/, const std::string& InPublicKey /*path to public key pem file*/, std::string& OutString /*ciphertext*/) {
    
    // Load key
    FILE* f = fopen(InPublicKey.c_str(), "r");
    EVP_PKEY* pkey = PEM_read_PUBKEY(f, NULL, NULL, NULL);
    fclose(f);
    
    // Create/initialize context
    EVP_PKEY_CTX* ctx;
    ctx = EVP_PKEY_CTX_new(pkey, NULL);
    EVP_PKEY_encrypt_init(ctx);

    // Specify padding: default is PKCS#1 v1.5
    // EVP_PKEY_CTX_set_rsa_padding(ctx, RSA_PKCS1_OAEP_PADDING); // for OAEP with SHA1 for both digests

    // Encryption
    size_t ciphertextLen;
    EVP_PKEY_encrypt(ctx, NULL, &ciphertextLen, (const unsigned char*)InStr.c_str(), InStr.size());
    unsigned char* ciphertext = (unsigned char*)OPENSSL_malloc(ciphertextLen);
    EVP_PKEY_encrypt(ctx, ciphertext, &ciphertextLen, (const unsigned char*)InStr.c_str(), InStr.size());
    OutString.assign((char*)ciphertext, ciphertextLen);

    // Release memory
    EVP_PKEY_free(pkey);
    EVP_PKEY_CTX_free(ctx);
    OPENSSL_free(ciphertext);

    return true; // add exception/error handling
}

With the ciphertext it must be considered that it is a byte sequence that does not represent real text (it may contain 0x00 values, for example). If the ciphertext is to be represented as real text, an additional binary-to-text encoding such as Base64 encoding must be performed.

The code requires a PEM encoded public key in X.509/SPKI format. If the public key has the PKCS#1 format, it can be imported using PEM_read_RSAPublicKey() and EVP_PKEY_set1_RSA().