Search code examples
pythonpython-3.xsocketstcp

Divide 'bytes' object into chunks in Python

I need to send an array by 'packages' of specific length (say 40 bytes) using TCP Protocol and Python's socket module. First I'm generating sample array of ints:

int_array = [i**2 for i in range(1, 100)]

Then I'm converting it into bytes object:

encoded_array = str(int_array).encode()

To check length of encoded_array object I'm using len(encoded_array) which in this particular case equals 551 bytes.

Say I want to sent packages that are 40 bytes long: this makes full 13 packages and the leftovers from last bytes of encoded_array, total of 14 packages. But I can't figure out the method how to divide bytes object into smaller bytes objects by number of bytes itself. Does anyone have an idea how to do this?

I'm also aware that I may be doing it in totally wrong way, so please advise me on how to send 'portioned' data via TCP protocol. Maybe there's no need to do this splitting at all?

Solution

  • bytes can be tricky.

    First off, encoded_array = str(int_array).encode() is not doing what you think. If you print encoded_array, you'll see that it's literally converting the to_string value of int_array to bytes. This is why the first value of encoded_array is [

    >>> encoded_array[:1]
b'['

    Second, I'm not sure you want int_array = [i**2 for i in range(1, 100)] to do what it's doing. It creates values up to 10,000. I'm unsure if you would like for this range to be between 0 and 256 if each element is to represent a byte. I'm going to assume you would like some data to be converted to bytes and split into 40 byte chunks and the data in this case is an array of integers.

    First let's convert your int_array into an array of bytes. I'm going to convert each int into a 2 byte value and represent it in hex.

    >>> hex_array = [x.to_bytes(2, byteorder="big") for x in int_array]

    Now to split up the data into bytes of 40

    >>> h
[]

>>> for x in range(0, int(round(len(hex_array)/20))):
...     h.append(hex_array[:20])
...     del hex_array[:20]

    I'm splitting by 20 because each element holds 2 bytes

    Now you'll have an array of max 40 byte collections. Time to join the collections together so we can transmit them!

    >>> result
[]

>>> for package in h:
...     joined_package = b';'.join(package)
...     result.append(joined_package)