How to deal with the sign bit of integer representations with odd bit counts?

Question

Let's assume we have a representation of -63 as signed seven-bit integer within a uint16_t. How can we convert that number to float and back again, when we don't know the representation type (like two's complement).

An application for such an encoding could be that several numbers are stored in one int16_t. The bit-count could be known for each number and the data is read/written from a third-party library (see for example the encoding format of tivxDmpacDofNode() here: https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/latest/exports/docs/tiovx/docs/user_guide/group__group__vision__function__dmpac__dof.html --- but this is just an example). An algorithm should be developed that makes the compiler create the right encoding/decoding independent from the actual representation type. Of course it is assumed that the compiler uses the same representation type as the library does.

One way that seems to work well, is to shift the bits such that their sign bit coincides with the sign bit of an int16_t and let the compiler do the rest. Of course this makes an appropriate multiplication or division necessary.

Please see this example:

#include <iostream>
#include <cmath>

int main()
{

    // -63 as signed seven-bits representation
    uint16_t data = 0b1000001;
    
    // Shift 9 bits to the left
    int16_t correct_sign_data = static_cast<int16_t>(data << 9);
    
    float f = static_cast<float>(correct_sign_data);
    
    // Undo effect of shifting
    f /= pow(2, 9);
    
    std::cout << f << std::endl;
    
    // Now back to signed bits
    f *= pow(2, 9);
    
    uint16_t bits = static_cast<uint16_t>(static_cast<int16_t>(f)) >> 9;
    
    std::cout << "Equals: " << (data == bits) << std::endl;
    
    return 0;
}

I have two questions:

1. This example uses actually a number with known representation type (two's complement) converted by https://www.exploringbinary.com/twos-complement-converter/. Is the bit-shifting still independent from that and would it work also for other representation types?
2. Is this the canonical and/or most elegant way to do it?

Clarification:

I know the bit width of the integers I would like to convert (please check the link to the TIOVX example above), but the integer representation type is not specified.

The intention is to write code that can be recompiled without changes on a system with another integer representation type and still correctly converts from int to float and/or back.

My claim is that the example source code above does exactly that (except that the example input data is hardcoded and it would have to be different if the integer representation type were not two's complement). Am I right? Could such a "portable" solution be written also with a different (more elegant/canonical) technique?

Answer 1

Your question is ambiguous as to whether you intend to truly store odd-bit integers, or odd-bit floats represented by custom-encoded odd-bit integers. I'm assuming by "not knowing" the bit-width of the integer, that you mean that the bit-width isn't known at compile time, but is discovered at runtime as your custom values are parsed from a file, for example.

---

Edit by author of original post:

The assumption in the original question that the presented code is independent from the actual integer representation type, is wrong (as explained in the comments). Integer types are not specified, for example it is not clear that the leftmost bit is the sign bit. Therefore the presented code also contains assumptions, they are just different (and most probably worse) than the assumption "integer representation type is two's complement".

---

Here's a simple example of storing an odd-bit integer. I provide a simple struct that let's you decide how many bits are in your integer. However, for simplicity in this example, I used uint8_t which has a maximum of 8-bits obviously. There are several different assumptions and simplifications made here, so if you want help on any specific nuance, please specify more in the comments and I will edit this answer.

One key detail is to properly mask off your n-bit integer after performing 2's complement conversions.

Also please note that I have basically ignored overflow concerns and bit-width switching concerns that may or may not be a problem depending on how you intend to use your custom-width integers and the maximum bit-width you intend to support.

#include <iostream>
#include <string>

struct CustomInt {
    int bitCount = 7;
    uint8_t value;
    uint8_t mask = 0;

    CustomInt(int _bitCount, uint8_t _value) {
        bitCount = _bitCount;
        value = _value;
        mask = 0;

        for (int i = 0; i < bitCount; ++i) {
            mask |= (1 << i);
        }
    }

    bool isNegative() {
        return (value >> (bitCount - 1)) & 1;
    }

    int toInt() {
        bool negative = isNegative();
        uint8_t tempVal = value;
        if (negative) {
            tempVal = ((~tempVal) + 1) & mask;
        }

        int ret = tempVal;
        return negative ? -ret : ret;
    }

    float toFloat() {
        return toInt(); //Implied truncation!
    }

    void setFromFloat(float f) {
        int intVal = f; //Implied truncation!
        bool negative = f < 0;
        if (negative) {
            intVal = -intVal;
        }

        value = intVal;
        if (negative) {
            value = ((~value) + 1) & mask;
        }
    }
};

int main() {
    CustomInt test(7, 0b01001110); // -50. Would be 78 if this were a normal 8-bit integer
    std::cout << test.toFloat() << std::endl;
}