"Malformed UTF-8 characters" error when trying to json_encode and array that holds some encrypted values

Question

I need to send some JSON data to an API endpoint that requires parts of the request to be encrypted. I have a public key that was supplied to me by the API provider. Here's the relevant piece of code:

$key = "-----BEGIN PUBLIC KEY-----
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
-----END PUBLIC KEY-----";

openssl_public_encrypt('my_username', $username, openssl_pkey_get_public($key));
openssl_public_encrypt('my_pa55w0rd', $password, openssl_pkey_get_public($key));

$client = new GuzzleHttp\Client();
$result = $client->post(
    'https://api.domain.com/endpoint',
    [
        'headers' => [
            'Authorization' => 'Bearer ' . $bearerToken,
            'Content-Type' => 'application/json',
        ],
        'json' => [
            'username' => $username,
            'pasword' => $password,
            'unencrypted_key' => 'an unencrypted value,
        ]
    ]
);

The code snippet above gives me an json_encode error: Malformed UTF-8 characters, possibly incorrectly encoded error. When doing a echo $username; I see that the string outputted has a bunch of malformed characters:

I'm not sure what I'm doing wrong or if there's some other approach I should take to encrypting the values before it gets json encoded.

NOTE: Although I am not using the json_encode function in my code above, I believe the guzzle HTTP library json_encodes the array before sending out the request.

Answer 1

As msg noted in a comment:

openssl_public_encrypt returns a binary stream, that's the mangled output, you would need to encode it in ascii first, probably with base64_encode but why do you need encryption if the transport is already using ssl ? Check with the api provider.

However, it's also possible that they're expecting hex-encoded data instead. Give bin2hex() a try instead.

If they have some other weird format (Base32Hex?), feel free to use this RFC 4648 library to encode your messages.

---

Important Security Information

This isn't an answer to your question, but it is something you and your API should both be aware of.

openssl_public_encrypt defaults to RSA with PKCS#1 v1.5 padding, which is vulnerable to a padding oracle attack.

At a minimum: The developer for the API you're communicating with should stop accepting data encrypted with RSA with PKCS#1v1.5 padding and instead only accept OAEP ciphertexts.

There are ways to work around Bleichenbacher's 1998 attack but they're messy and can only be solved at a protocol level, not a the library or primitive level.

However, RSA encryption is messy for other reasons: You can't encrypt large messages directly with RSA.

There are also plenty of other ways RSA can go wrong.

Recommendation

A better solution is to stop using RSA entirely.

Libsodium has bindings in most popular programming languages (and is included in the PHP 7.2 core) and makes this easier to get right.

// On your end...
$sendToProvider = sodium_bin2hex(
    sodium_crypto_box_seal($privateData, $publicKey)
);

// On their end...
$decrypted = sodium_crypto_box_seal_open(
    sodium_hex2bin($encrypted),
    $keypair
);

You would have encountered the encoding issue either way (since both APIs return raw binary strings). However, using libsodium would have side-stepped a ton of security issues you probably weren't aware of until you read this StackOverflow answer.

Also: Don't feel bad if you didn't know. You're not alone. Even Zend Framework's cryptography library got bit by these RSA vulnerabilities, and they had Enrico Zimuel on their team. RSA is just a bad choice for real world cryptography.