Search code examples
c#imagepixeldirect2dslimdx

Convert Pixel Buffer to B8G8R8A8_UNorm from 16Bit


So I have, from an external native library (c++) a pixel buffer that appears to be in 16Bit RGB (SlimDX equivalent is B5G6R5_UNorm).

I want to display the image that is represented by this buffer using Direct2D. But Direct2D does not support B5G6R5_UNorm.

so I need to convert this pixel buffer to B8G8R8A8_UNorm

I have seen various code snippets of such a task using bit shifting methods, but none of which were specific for my needs or formats. It doesn't help i have zero, nada, none, zilch any clue about bit shifting, or how it is done.

What i am after is a C♯ code example of such a task or any built in method to do the conversion - I don't mind using other library's

Please Note : I know this can be done using the C♯ bitmap classes, but i am trying to not rely on these built in classes (There is something about GDI i don't like), the images (in the form of pixel buffers) will be coming in thick and fast, and i am choosing SlimDX for its ease of use and performance.

The reason why I believe I need to convert the pixel buffer is, if I draw the image using B8G8R8A8_UNorm the image has a green covering and the pixels are just all over the place, hence why i believe i need to first convert or 'upgrade' the pixel buffer to the required format.

Just to add : When i do the above without converting the buffer, the image doesn't fill the entire geometry.

The pixel buffers are provided via byte[] objects


Solution

  • Bit shifting and logical operators are really useful when dealing with image formats, so you owe it to yourself to read more about it. However, I can give you a quick run-down of what this pixel format represents, and how to convert from one to another. I should preface my answer with a warning that I really don't know C# and its support libraries all that well, so there may be an in-box solution for you.

    First of all, your pixel buffer has the format B5G6R5_UNORM. So we've got 16 bits (5 red, 6 green, and 5 blue) assigned to each pixel. We can visualize the bit layout of this pixel format as "RRRRRGGGGGGBBBBB", where 'R' stands for bits that belong to the red channel, 'G' for bits that belong to the green channel, and 'B' for bits that belong to the blue channel.

    Now, let's say the first 16 bits (two bytes) of your pixel buffer are 1111110100101111. Line that up with the bit layout of your pixel format...

    RRRRRGGGGGGBBBBB
    1111110100101111
    

    This means the red channel has bits 11111, green has 101001, and blue has 01111. Converting from binary to decimal: red=31, green=41, and blue=15. You'll notice the red channel has all bits set 1, but its value (31) is actually smaller than the green channel (41). However, this doesn't mean the color is more green than red when displayed; the green channel has an extra bit, so it can represent more values than the red and blue channels, but in this particular example there is actually more red in the output color! That's where the UNORM part comes in...

    UNORM stands for unsigned normalized integer; this means the color channel values are to be interpreted as evenly spaced floating-point numbers from 0.0 to 1.0. The values are normalized by the number of bits allocated. What does that mean, exactly? Let's say you had a format with only 3 bits to store a channel. This means the channel can have 2^3=8 different values, which are shown below with the respective decimal, binary, and normalized representations. The normalized value is just the decimal value divided by the largest possible decimal value that can be represented with N bits.

    Decimal | Binary | Normalized
    -----------------------------
    0       | 000    | 0/7 =  0.000
    1       | 001    | 1/7 =~ 0.142
    2       | 010    | 2/7 =~ 0.285
    3       | 011    | 3/7 =~ 0.428
    4       | 100    | 4/7 =~ 0.571
    5       | 101    | 5/7 =~ 0.714
    6       | 110    | 6/7 =~ 0.857
    7       | 111    | 7/7 =  1.000
    

    Going back to the earlier example, where the pixel had bits 1111110100101111, we already know our decimal values for the three color channels: RGB = {31, 41, 15}. We want the normalized values instead, because the decimal values are misleading and don't tell us much without knowing how many bits they were stored in. The red and blue channels are stored with 5 bits, so the largest decimal value is 2^5-1=31; however, the green channel's largest decimal value is 2^6-1=63. Knowing this, the normalized color channels are:

    // NormalizedValue = DecimalValue / MaxDecimalValue
    R = 31 / 31 =  1.000
    G = 41 / 63 =~ 0.650
    B = 15 / 31 =~ 0.483
    

    To reiterate, the normalized values are useful because they represent the relative contribution of each color channel in the output. Adding more bits to a given channel doesn't affect the range of possible color, it simply improves color accuracy (more shades of that color channel, basically).

    Knowing all of the above, you should be able to convert from any RGB(A) format, regardless of how many bits are stored in each channel, to any other RGB(A) format. For example, let's convert the normalized values we just calculated to B8G8R8A8_UNORM. This is easy once you have normalized values calculated, because you just scale by the maximum value in the new format. Every channel uses 8 bits, so the maximum value is 2^8-1=255. Since the original format didn't have an alpha channel, you would typically just store the max value (meaning fully opaque).

    // OutputValue = InputValueNormalized * MaxOutputValue
    B = 0.483 * 255 = 123.165
    G = 0.650 * 255 = 165.75
    R = 1.000 * 255 = 255
    A = 1.000 * 255 = 255
    

    There's only one thing missing now before you can code this. Way up above, I was able to pull out the bits for each channel just by lining up them up and copying them. That's how I got the green bits 101001. In code, this can be done by "masking" out the bits we don't care about. Shifting does exactly what it sounds like: it moves bits to the right or left. When you move bits to the right, the rightmost bit gets discarded and the new leftmost bit is assigned 0. Visualization below using the 16 bit example from above.

    1111110100101111 // original 16 bits
    0111111010010111 // shift right 1x
    0011111101001011 // shift right 2x
    0001111110100101 // shift right 3x
    0000111111010010 // shift right 4x
    0000011111101001 // shift right 5x
    

    You can keep shifting, and eventually you'll end up with sixteen 0's. However, I stopped at five shifts for a reason. Notice now the 6 rightmost bits are the green bits (I've shifted/discarded the 5 blue bits). We've very nearly extracted the exact bits we need, but there's still the extra 5 red bits to the left of the green bits. To remove these, we use a "logical and" operation to mask out only the rightmost 6 bits. The mask, in binary, is 0000000000111111; 1 means we want the bit, and 0 means we don't want it. The mask is all 0's except for the last 6 positions, because we only want the last 6 bits. Line this mask up with the 5x shifted number, and the output is 1 when both bits are 1, and 0 for every other bit:

    0000011111101001 // original 16 bits shifted 5x to the right
    0000000000111111 // bit mask to extract the rightmost 6 bits
    ------------------------------------------------------------
    0000000000101001 // result of the 'logical and' of the two above numbers
    

    The result is exactly the number we're looking for: the 6 green bits and nothing else. Recall that the leading 0's have no effect on the decimal value (it's still 41). It's very simple to do the 'shift right' (>>) and 'logical and' (&) operations in C# or any other C-like language. Here's what it looks like in C#:

    // 0xFD2F is 1111110100101111 in binary
    uint pixel = 0xFD2F;
    
    // 0x1F is 00011111 in binary (5 rightmost bits are 1)
    uint mask5bits = 0x1F;
    
    // 0x3F is 00111111 in binary (6 rightmost bits are 1)
    uint mask6bits = 0x3F;
    
    // shift right 11x (discard 5 blue + 6 green bits), then mask 5 bits
    uint red   = (pixel >> 11) & mask5bits;
    
    // shift right 5x (discard 5 blue bits), then mask 6 bits
    uint green = (pixel >> 5)  & mask6bits;
    
    // mask 5 rightmost bits
    uint blue  = pixel & mask5bits;
    

    Putting it all together, you might end up with a routine that looks similar to this. Do be careful to read up on endianness, however, to make sure the bytes are ordered in the way you expect. In this case, the parameter is a 32-bit unsigned integer (first 16 bits ignored)

    byte[] R5G6B5toR8G8B8A8(UInt16 input)
    {
        return new byte[]
        {
            (byte)((input & 0x1F) / 31.0f * 255),         // blue
            (byte)(((input >> 5) & 0x3F) / 63.0f * 255),  // green
            (byte)(((input >> 11) & 0x1F) / 31.0f * 255), // red
            255                                           // alpha
        };
    }