Search code examples
pythonpython-3.xnumpybit-manipulationendianness

Numpy pack bits into 32-bit little-endian values


Numpy provides packbits function to convert from values to individual bits. With bitorder='little' I can read them in C as uint8_t values without issues. However, I would like to read them as uint32_t values. This means that I have to reverse the order of each 4 bytes. I tried to use

import numpy as np

array = np.array([1,0,1,1,0,1,0,1,0,1,0,0,1,0,1,1,0,0,1,1,0,1,0,1,1,0,0,1,0,1,0,1, 
   1,0,0,1,1,0,1,0,1,1,0,0,1,1,1,0,0,1])
array = np.packbits(array, bitorder='little')
array.dtype = np.uint32
array.byteswap(inplace=True)

print(array)

but have the following error:

Traceback (most recent call last):
  File "sample.py", line 5, in <module>
    array.dtype = np.uint32
ValueError: When changing to a larger dtype, its size must be a divisor of the total size in bytes of the last axis of the array.

I have 50 bits in the input. The first chunk of 32 bits written in the little-endian format (earliest input bit is the least significant bit) are 0b10101001101011001101001010101101 = 2846675629, the second is 0b100111001101011001 = 160601. So the expected output is

[2846675629 160601]

Solution

  • My first answer fixes the exception.

    This answer, relies on this and this

    • Pad the array from the right to the nearest power of 2
    • Reshape to have some arrays, each array of size 32
    • Pack bits PER ARRAY and only then view as unit32.
    import numpy as np
    import math
    
    
    # https://stackoverflow.com/questions/49791312/numpy-packbits-pack-to-uint16-array
    # https://stackoverflow.com/questions/36534035/pad-0s-of-numpy-array-to-nearest-power-of-two/36534077
    
    
    def next_power_of_2(number):
        # Returns next power of two following 'number'
        return 2**math.ceil(math.log(number, 2))
    
    
    a = np.array([
        1, 0, 1, 1, 0, 1, 0, 1, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 1, 0, 1, 0, 1,
        1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1
    ])
    
    # a = np.array([
    #     0 for _ in range(31)
    # ] + [1])
    padding_size = next_power_of_2(len(a)) - len(a)
    b = np.concatenate([a, np.zeros(padding_size)])
    c = b.reshape((-1, 32)).astype(np.uint8)
    d = np.packbits(c, bitorder='little').view(np.uint32)
    
    print(d)
    

    output:

    [2846675629 160601]