Search code examples
pythonstringlistpackunpack

convert list of integers to bytes/ascii string and back ? Fast?


I'm trying to convert list of numbers to string and reverse.. here is what i got

def nums2ascii(nums,size=2):
    return b''.join([n.to_bytes(size,'big') for n in nums ])


def ascii2nums(ascii,size=2):
    return [int.from_bytes(ascii[i:i+size],'big') for i in range(0,len(ascii),size)] 

first problem is that this is the slowest method

I read that the way to go is to use struct module to make it fast. there are too many mnemonics, how do i do it simply 2,4,8. Do i read it correctly '>I', '>R', '>Q' ? Can I just pass the list to pack/unpack directly instead of doing list-comprehension ?

My second question is how transparently to handle int8, int16,, int32,int64 ... do i stick with the largest one OR do I make a decision to stick with specific one across the app /in which case i have to put an assert !!


Solution

  • I would do something like this, assuming the system short "h" is two bytes (it usually is):

    import struct
    
    def nums2bytes(nums):
        return struct.pack(f">{len(nums)}h", *nums)
    
    def bytes2nums(b):
        return struct.unpack(f">{len(b)//2}h", b)
    

    You could maybe build out functionality that says that 2 is "h", 4 is "i", etc. Note that capital letters denote unsigned integers, lowercases denote signed.

    An alternative is to install numpy and use numpy arrays.