Python sockets in a multithreaded program with classes/objects interfere with each other
As an aid to my work, I am developing a communication simulator between devices, based on TCP sockets, in Python 3.12 (with an object oriented approach).
Basically, the difference between a SERVER type communication channel rather than a CLIENT type is merely based on the way by which the sockets are instantiated: respectively, the server ones listen/accept for connection requests while the client ones actively connect to their endpoint.
Once the connection is established, either party can begin to transmit something, which the other receives and processes and then responds (this on the same socket pair of course).
As you can see. this simulator has a simple interface based on Tkinter
You can create up to 4 channels n a grid layout, in this case we have two:
When the user clicks on CONNECT button, this is what happens in the listener of that button in the frame class:
class ChannelFrame(tk.Frame):
channel = None #istance of channel/socket type
def connectChannel(self):
port = self.textPort.get();
if self.socketType.get() == 'SOCKET_SERVER':
self.channel = ChannelServerManager(self,self.title,port)
elif self.socketType.get() == 'SOCKET_CLIENT':
ipAddress = self.textIP.get()
self.channel = ChannelClientManager(self,self.title,ipAddress,port)
Then I have an implementation of a channel of type Server and one for type Client. Their constructors basically collect the received data and create a main thread whose aim is to create socket and then:
1a) connect to the counterpart in case of socket client
1b) waiting for requests of connections in case of socket server
2.) enter a main loop using select.select and trace in the text area of their frame the received and sent data
Here is the code for main thread Client
class ChannelClientManager():
establishedConn = None
receivedData = None
eventMainThread = None #set this event when user clicks on DISCONNECT button
def threadClient(self):
self.socketsInOut.clear()
self.connected = False
while True:
if (self.eventMainThread.is_set()):
print(f"threadClient() --> ChannelClient {self.channelId}: Socket client requested to shut down, exit main loop")
break;
if(not self.connected):
try :
self.establishedConn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.establishedConn.connect((self.ipAddress, int(self.port)))
self.channelFrame.setConnectionStateChannel(True)
self.socketsInOut.append(self.establishedConn)
self.connected = True
#keep on trying to connect to my counterpart until I make it
except socket.error as err:
print(f'socket.error threadClient() --> ChannelClient {self.channelId}: Error while connecting to server: {err}')
time.sleep(0.5)
continue
except socket.timeout as sockTimeout:
print(f'socket.timeout threadClient() --> ChannelClient {self.channelId}: Timeout while connecting to server: {sockTimeout}')
continue
except Exception as e:
print(f'Exception on connecting threadClient() --> ChannelClient {self.channelId}: {e}')
continue
if(self.connected):
try:
r, _, _ = select.select(self.socketsInOut, [], [], ChannelClientManager.TIMEOUT_SELECT)
if len(r) > 0: #socket ready to be read with incoming data
for fd in r:
data = fd.recv(1)
if data:
self.manageReceivedDataChunk(data)
else:
print(f"ChannelClient {self.channelId}: Received not data on read socket, server connection closed")
self.closeConnection()
else:
#timeout
self.manageReceivedPartialData()
except ConnectionResetError as crp:
print(f"ConnectionResetError threadClient() --> ChannelClient {self.channelId}: {crp}")
self.closeConnection()
except Exception as e:
print(f'Exception on selecting threadClient() --> ChannelClient {self.channelId}: {e}')
Here is the code for main thread Server
class ChannelServerManager():
socketServer = None #user to listen/accept connections
establishedConn = None #represents accepted connections with the counterpart
receivedData = None
eventMainThread = None
socketsInOut = []
def __init__(self, channelFrame, channelId, port):
self.eventMainThread = Event()
self.socketsInOut.clear()
self.socketServer = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socketServer.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.socketServer.bind(('', int(port))) #in ascolto qualsiasi interfaccia di rete, se metto 127.0.0.1 starebbe in ascolto solo sulla loopback
self.socketServer.listen(1) #accepting one connection from client
self.socketsInOut.append(self.socketServer)
self.mainThread = Thread(target = self.threadServer)
self.mainThread.start()
def threadServer(self):
self.receivedData = ''
while True:
if (self.eventMainThread.is_set()):
print("threadServer() --> ChannelServer is requested to shut down, exit main loop\n")
break;
try:
r, _, _ = select.select(self.socketsInOut, [], [], ChannelServerManager.TIMEOUT_SELECT)
if len(r) > 0: #socket pronte per essere lette
for fd in r:
if fd is self.socketServer:
#if the socket ready is my socket server, then we have a client wanting to connect --> let's accept it
clientsock, clientaddr = self.socketServer.accept()
self.establishedConn = clientsock
print(f"ChannelServer {self.channelId} is connected from client address {clientaddr}")
self.socketsInOut.append(clientsock)
self.channelFrame.setConnectionStateChannel(True)
self.receivedData = ''
elif fd is self.establishedConn:
data = fd.recv(1)
if not data:
print(f"ChannelServer {self.channelId}: Received not data on read socket, client connection closed")
self.socketsInOut.remove(fd)
self.closeConnection()
else:
self.manageReceivedDataChunk(data)
else: #timeout
self.manageReceivedPartialData()
except Exception as e:
print(f"Exception threadServer() --> ChannelServer {self.channelId}: {traceback.format_exc()}")
I don't know why, but this frames/sockets appear to interfere with each other or "share data". Or, disconnecting and closing a channel from its button in its own frame also causes the other one into error, or the other one closes/crashes too. These two frames/objects should each live their own life and move forward with their counterpart as long as it is connected, instead they interfere. As you can see from this screenshot:
By a medical device (which is server), I am sending this data
<VT>MSH|^~\&|KaliSil|KaliSil|AM|HALIA|20240130182136||OML^O33^OML_O33|1599920240130182136|P|2.5<CR>PID|1||A20230522001^^^^PI~090000^^^^CF||ESSAI^Halia||19890522|M|||^^^^^^H|||||||||||||||<CR>PV1||I||||||||||||A|||||||||||||||||||||||||||||||<CR>SPM|1|072401301016^072401301016||h_san^|||||||||||||20240130181800|20240130181835<CR>ORC|NW|072401301016||A20240130016|saisie||||20240130181800|||^^|CP1A^^^^^^^^CP1A||20240130182136||||||A^^^^^ZONA<CR>TQ1|1||||||||0||<CR>OBR|1|072401301016||h_GLU_A^^T<CR>OBX|1|NM|h_GLU_A^^T||||||||||||||||<CR>BLG|D<CR><FS>
only to channel on port 10001 but part of this data is received on one socket client, other part on the other (right) socket client. This is not a problem of rendering the text in the right frame, also the log of the received data shows that some data is received in Channel 0 and some other data in Channel 1.
Why does this happen? Instead, I start 2 instances of the simulator with only one channel each, then everything works perfectly but this defeats our purpose of being able to work up to 4 channels in parallel from a single window.
Do you have any ideas? The first time I had implemented ChannelServerManager and ChannelClientManager as extended from an ChannelAbstractManager with common methods and data structures, based on Python library ABC
Then I read that inheritance in Python is not the same as in Java, so I thought the different instances were sharing some attributes. I removed the abstract class and replicated
the code and resources in both classes but this has not solved.
Any suggestions?
Then I read that inheritance in Python is not the same as in Java
Thanks, that was a good tip to find the issue! While not an inheritance issue, I think you're seeing problems caused by this Java-ism:
class ChannelClientManager():
establishedConn = None
receivedData = None
eventMainThread = None #set this event when user clicks on DISCONNECT button
...
class ChannelServerManager():
socketServer = None #user to listen/accept connections
establishedConn = None #represents accepted connections with the counterpart
receivedData = None
eventMainThread = None
socketsInOut = []
def __init__(self, channelFrame, channelId, port):
...
In Python you don't need to declare your attributes in advance, you should just assign to them directly in your __init__ method (constructor). All these variables you're declaring are actually class attributes and hence shared between all instances like you suspected.
It might not be obvious because when you do self.establishedConn = ... you're also creating an instance attribute that overrides the visibility of the class attribute, so you never actually access the shared values.
With one exception:
socketsInOut = []
def __init__(self, channelFrame, channelId, port):
...
self.socketsInOut.clear()
...
self.socketsInOut.append(self.socketServer)
Because you never assigned self.socketsInOut, all instances are instead accessing the (shared) class attribute. After a small startup period, all channels end up sending and receiving messages on the same socket list, hence the messages being split.
The fix is to remove all those unnecessary attribute "declarations", and add a missing one for socketsInOut:
class ChannelClientManager():
def threadClient(self):
self.socketsInOut = [] # Change here.
...
class ChannelServerManager():
def __init__(self, channelFrame, channelId, port):
self.eventMainThread = Event()
self.socketsInOut = [] # And here.
self.socketServer = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
...
And to save you time debugging another well known Python wart, default arguments are also shared between calls. So never declare a method with a mutable default argument, like def foo(items=[]): ....