Python socket receiving multiple messages at the same time

Question

I'm making a TCP/IP chat with python (3) sockets, and I have been having the same problem over multiple instances of socket.send/socket.receive pairs. As an example:

Everytime the server updates the list of connected clients, it first sends a string signaling that it is about to do so, sends the list item by item, and sends another string signal to say it is done. Client-side, I have a thread that receives messages and handles them, and it has a specific case for dealing with this specific string signal. In this case, it starts a loop to receive client names until it receives a signal saying that the list of clients is over.

Often, though not always, either the client names or the string signals or both get mixed up as a single-message, however. If I have clients C1, C2, C3 and send the signal "over" to say the list is done, my list might display something like:

C1C2
C3
"over"

Since client-side only has this thread and GUI/Main thread, and server-side no other type of message gets mixed up (on threads for listening, handling clients and GUI/Main), I'm assuming it isn't a synchronization issue. I've tried adding time.sleep() funcs of varied size between the signals and the list, but it still happens.

I have noticed this throughout my entire experience with this socket chat, but had been able to figure out fixes (usually with sleep()), but this one has me stumped. Am I doing something fundamentally wrong that is messing up my sending and receiving of messages? How can I guarantee that a single piece of data will be sent at each socket.send()?

Answer 1

TCP is a byte-stream protocol. There are no messages but just a bunch of bytes coming in. You must implement a protocol and buffer the data received until you know you have a complete message.

You can use the built-in socket.makefile() method to implement a line-oriented protocol. Example:

server.py

from socket import *

s = socket()
s.bind(('',5000))
s.listen(1)

while True:
    c,a = s.accept()
    print(f'connect: {a}')
    read  = c.makefile('r')
    write = c.makefile('w')

    with c,read,write:
        while True:
            data = read.readline()
            if not data: break
            cmd = data.strip()
            print(f'cmd: {cmd}')
            if cmd == 'LIST':
                write.write('C1\nC2\nC3\nDONE\n')
                write.flush()

    print(f'disconnect: {a}')

client.py

from socket import *

s = socket()
s.connect(('localhost',5000))
read = s.makefile('r',)
write = s.makefile('w')

def send(cmd):
    print(cmd)
    write.write(cmd + '\n')
    write.flush()

with s,read,write:
    send('TEST')
    send('LIST')
    while True:
        data = read.readline()
        if not data: break
        item = data.strip()
        if item == 'DONE': break
        print(f'item: {item}')
    send('OTHER')

Server Output:

connect: ('127.0.0.1', 13338)
cmd: TEST
cmd: LIST
cmd: OTHER
disconnect: ('127.0.0.1', 13338)

Client Output:

TEST
LIST
item: C1
item: C2
item: C3
OTHER