Search code examples
c++crypto++

Crypto++ HexEncoder not working consistently


Here's my code

#include <cryptopp/hex.h>

#include <string>
#include <iostream>

void hexlify(CryptoPP::byte* bytes, std::string &hex_string, size_t size)
{
    CryptoPP::StringSource ss(bytes, size, true, new CryptoPP::HexEncoder(new CryptoPP::StringSink(hex_string)));
}

void unhexlify(std::string hex_string, CryptoPP::byte* &bytes)
{
    std::string decoded;

    CryptoPP::StringSource ss(hex_string, true, new CryptoPP::HexDecoder(new CryptoPP::StringSink(decoded)));

    std::cout << decoded + "\n"; // For testing
    bytes = (CryptoPP::byte*)decoded.data();
}

int main()
{
    std::string seed = "BF0F3123B21A60E5AB7615AD06EA16A2BD44D84CED4DCC10AA0413127F87DC60";
    std::cout << "\n" + seed + "\n";

    CryptoPP::byte* b;
    unhexlify(seed, b);

    std::string s;
    hexlify(b, s, 32);

    std::cout << s;
    std::cout << "\n\n";
}

It should take the 32-byte, 64-character hex string seed, print it, convert it into bytes, then convert it back into a hex string (s) and print it. I also have it print the decoded string in the unhexlify function.

My expectation is for the output to look like this:

BF0F3123B21A60E5AB7615AD06EA16A2BD44D84CED4DCC10AA0413127F87DC60
?1#?`?v????D?L?M????`
BF0F3123B21A60E5AB7615AD06EA16A2BD44D84CED4DCC10AA0413127F87DC60

The first line (seed) and the third line (s, the seed converted into bytes and back) should be the same. And very rarely, that's exactly what happens. However, most of the time the first and third lines are completely different.

What's weird is that I made zero changes to the code, and I didn't even recompile it. But every time I run the executable, it says something different. What's even weirder is that the third line is different every time I run it. Again, it happens even though I'm running the same executable without changing the code or recompiling. Here are the outputs from 5 runs:

BF0F3123B21A60E5AB7615AD06EA16A2BD44D84CED4DCC10AA0413127F87DC60
?1#?`?v????D?L?M????`
205BC0AE8B7F0000AB7615AD06EA16A2889A960301000000189C960301000000
BF0F3123B21A60E5AB7615AD06EA16A2BD44D84CED4DCC10AA0413127F87DC60
?1#?`?v????D?L?M????`
205B40C2B87F0000AB7615AD06EA16A288FA9B060100000018FC9B0601000000
BF0F3123B21A60E5AB7615AD06EA16A2BD44D84CED4DCC10AA0413127F87DC60
?1#?`?v????D?L?M????`
205BC07DA67F0000AB7615AD06EA16A2885A6C0B01000000185C6C0B01000000
BF0F3123B21A60E5AB7615AD06EA16A2BD44D84CED4DCC10AA0413127F87DC60
?1#?`?v????D?L?M????`
205BC06EB47F0000AB7615AD06EA16A288DAAE090100000018DCAE0901000000
BF0F3123B21A60E5AB7615AD06EA16A2BD44D84CED4DCC10AA0413127F87DC60
?1#?`?v????D?L?M????`
205BC0F9EB7F0000AB7615AD06EA16A288DAF5010100000018DCF50101000000

The second line is the same every time, so I know the problem is with the hexlify function and/or the Crypto++ HexEncoder. Also, the section AB7615AD06EA16A2 always matches in both seed and s. The section that does match goes from the 17th character to the 32nd character. That always happens for every seed I try. But everything else is usually different.

I don't understand why it's not encoding the bytes into a hex string properly. And I don't understand why it rarely does work even if I made no changes to the code. And I especially don't understand how the output can be different every time I run the same compiled code. What is going on here?


Solution

  • void unhexlify(std::string hex_string, CryptoPP::byte* &bytes)
    {
        std::string decoded;
    
        CryptoPP::StringSource ss(hex_string, true, new CryptoPP::HexDecoder(new CryptoPP::StringSink(decoded)));
    
        std::cout << decoded + "\n"; // For testing
        bytes = (CryptoPP::byte*)decoded.data();  // <--
    }
    

    You return the pointer to the buffer owned by the local variable decoded that dies after function exit. You get the undefined behavior with following passing the dangling pointer to hexlify. To fix the issue return string from std::string unhexlify(std::string hex_string) and call hexlify(bytes.data(), s. bytes.size()).