Search code examples
pythonlist-comprehension

Split a large object into constant size list entries via list comprehension


How to split a bytes object into a list/tuple of constant size objects? Ignore padding. Something like

max_size = 42
def foo(b: bytes):
    return [b[i:j] for (i, j) in range(max_size)]
foo(b'a' * 100000)

but working.

The 'list comprehension' part is only because it's concise, readable and efficient. IMHO a for() loop has been ugly for a decade or two.

Desired output: list[bytes objects of specific length]


Solution

  • I've provided a List comprehension and normal for loop solution.

    sample_bytes = b'Some string encoded as bytes'
    MAX_CHUNK_SIZE = 2
    
    # personally I find this much more readable than list comprehension
    def chunk_bytes_with_for_loop(b: bytes):
        for index in range(0, len(b), MAX_CHUNK_SIZE):
            yield b[index:index + MAX_CHUNK_SIZE] 
    
    def chunk_bytes_with_list_comprehension(b: bytes):
        return [b[index * MAX_CHUNK_SIZE:(index + 1) * MAX_CHUNK_SIZE] for index in range((len(b) + MAX_CHUNK_SIZE - 1) // MAX_CHUNK_SIZE )]
        
    print(list(chunk_bytes_with_for_loop(sample_bytes)))
    print(chunk_bytes_with_list_comprehension(sample_bytes))