I have a client-server code. On the client side, I want to send a list with = 200000 such as:
my_list = [1] * 200000
by socket programming. However, I do not want to split the list. All the lists should be sent at once.
This is my client code:
packedMylist = b''.join(struct.pack('256s', item.encode('utf-8')) for item in my_list)
Msg.append(struct.pack(f'{len(packedMylist)}s', packedMylist))
client.send(Msg)
The server code:
Msg = conn.recv(25600000)
packedMylist = struct.unpack(f'{len(Msg)', Msg)
My_list = [struct.unpack_from('256s', packedMylist, i*256)[0].decode('utf-8').rstrip('\x00') for i in range(200000)]
But it does not work properly. It generates error like this:
struct.error: unpack_from requires a buffer of at least 98048 bytes for unpacking 256 bytes at offset 97792 (actual buffer size is 97984)
First, decide what the elements of the list are. For struct.pack, s expects a bytes object where each element is a value 0-255. If you have a list of integers, i might be more appropriate (4-byte signed integer).
I'll assume you want integers.
Here's a server:
import socket
import struct
with socket.socket() as s:
# set up server
s.bind(('localhost', 5000))
s.listen()
# handle one connection at a time
while True:
client, addr = s.accept()
# Wrap the client in a file-like object to buffer the data.
# With recv(n), n is a *maximum* to receive, but you must check
# the return value to see what you really get.
# The file-like wrapper should be checked as well in case the socket
# closes prematurely, but assuming the data was sent correctly it will
# buffer data and receive what you request.
with client, client.makefile('rb') as rfile:
# multiple messages could be sent
while True:
header = rfile.read(4) # start with 4-byte # of elements in list
if not header:
break # client hung up
datalen, = struct.unpack('!L', header) # elements in the list
# list elements assumed to be 4-byte signed integers (type i)
data = struct.unpack(f'!{datalen}i', rfile.read(datalen * 4))
print(f'{datalen=} data=[{data[0]}, ..., {data[-1]}]')
And a simple client:
import socket
import struct
with socket.socket() as s:
s.connect(('localhost', 5000))
my_list = list(range(200000)) # [0, 1, ..., 199999]
# pack the number of elements and the entire list.
data = struct.pack(f'!L{len(my_list)}i', len(my_list), *my_list)
# Use sendall() to ensure all bytes are sent; otherwise, you must
# check the return value of send() because it may not send all bytes
# requested on one send.
s.sendall(data)
Start the server and run the client a few times.
Server output:
datalen=200000 data=[0, ..., 199999]
datalen=200000 data=[0, ..., 199999]
datalen=200000 data=[0, ..., 199999]