Coinbase FIX Logon signature

Question

Problem: I cannot properly encode signature, according to https://docs.cloud.coinbase.com/exchange/docs/messages . No matter what I'm doing, I'm getting "Invalid signature" rejection.

I'm using QuickFix FIX engine, and my code is written in C++. Signature calculation code provided below. b64_encode and hmac_sha256 are based on OpenSSL functions and were verified in another parts of application, so they are expected to be working correctly (unless Coinbase actually expects another kind of encryption).

const auto join_elem = '\x01';
auto data_to_encrypt =
    header.getField(FIX::FIELD::SendingTime) + join_elem +
    "A" + join_elem +
    header.getField(FIX::FIELD::MsgSeqNum) + join_elem +
    header.getField(FIX::FIELD::SenderCompID) + join_elem +
    header.getField(FIX::FIELD::TargetCompID) + join_elem +
    api_password;

std::vector<unsigned char> prehash(data_to_encrypt.begin(), data_to_encrypt.end());
auto signature_value = b64_encode(hmac_sha256(data, b64_decode(secret)));

As I'm using sandbox account, it is safe to share all the keys for the case if someone could verify calculation.
api_password=uxu3qgheh0e
SenderCompID=b0744eb59b6951d7955e5e5aaed3709f
secret=T/SabUfV28tyVDVv5AsMuk5oqDFdvGwQrfc6f1/RnatGfpAcpAMMRzjfON8nqoL6bAsUI6jkN9gWEX809z1iiA==

Complete logon example:
8=FIX.4.2|9=172|35=A|34=1|49=b0744eb59b6951d7955e5e5aaed3709f|52=20211112-16:45:45.856|56=Coinbase|96=6fsAC9XcV9Fm4II89s8VZgm4wmeiNclr3YY+mNTbMdo=|98=0|108=30|141=Y|554=uxu3qgheh0e|8013=Y|10=253|
And the following rejection:
8=FIX.4.2|9=128|35=3|34=1|49=Coinbase|52=20211112-16:45:46.985|56=b0744eb59b6951d7955e5e5aaed3709f|45=1|58=invalid signature|371=96|372=A|373=5|10=216|

Unfortunately, support replied with "Coinbase will only offer self-help customer support in your region", so I have to ask for help here. Thanks!

Answer 1

During a day of research, I finally found the issue. As I first step, I found working library written in NodeJS (it was easier to use it for testing) to ensure my credentials are working. Then I added console prints to NodeJS lib to see what values are calculated on the each step of signature making. Then I used the same values for my code and finally came to base64 decoding function, which I used to "decode" secret key. It was the following:

std::vector<unsigned char> b64_decode(const std::string& data) {
    BIO* b64 = BIO_new(BIO_f_base64());
    BIO_set_flags(b64, BIO_FLAGS_BASE64_NO_NL);

    BIO* bmem = BIO_new_mem_buf((void*)data.c_str(), data.length());
    bmem = BIO_push(b64, bmem);

    std::vector<unsigned char> output(data.length());
    int decoded_size = BIO_read(bmem, output.data(), output.size());
    BIO_free_all(bmem);

    return output;
}

This function appears to be incorrect if source data size not equal to output size, as I forgot to resize output according to actual length, and it was containing extra zeroes which caused decoded secret key to be incorrect. I.e. I just needed to add resize:

if (output.size() > decoded_size) {
    output.resize(decoded_size);
}