Is there a type of stream that is recommended for multithreading communication without losing data?

Question

I have a client server architecture that connects c# (unity) with python. I need to start multiple threads(multiple connections) on c# that will receive some data that will be deserialized from python server.

client(c# unity3d) -> 6 threads in c# will receive data from 6 processes that will serve data from python that will be deserialized.

After some debugging i managed to see that the packages are send in a good manner from the python server.

However on c# the packages on different threads are either with too less data either with too much. My assumption is that the socket from different threads are interfering when receiving data.

C# code for connector

public class Connector
{
public string ip = "127.0.0.1";

int bytesRead;
IPAddress ipAddress;
IPHostEntry ipHostInfo;

Socket sender;

int bytesRec;

[SerializeField]
private int dataOut;
public Connector() { }
public void establishConnection(int port)
{
    ipHostInfo = Dns.GetHostEntry(Dns.GetHostName());
    ipAddress = ipHostInfo.AddressList[0];
    sender = new Socket(ipAddress.AddressFamily,
         SocketType.Stream, ProtocolType.Tcp);

    sender.Connect(ip,port);
}


public void Send(byte[] input) {
    int bytesSent = sender.Send(input);

}

public byte[] Receive(int index) {

    byte[] bytesToRead = new byte[1024 * 1024 * 60];
    Array.Clear(bytesToRead, 0, bytesToRead.Length);

    try
    {
        bytesRec = sender.Receive(bytesToRead);
        Array.Resize(ref bytesToRead, bytesRec);
    }
    catch (Exception e)
    {
        Console.WriteLine(e.ToString());
    }

    if (bytesRec != 0)
    {
        return bytesToRead;
    }
    else
    {
        return null;
    }
}
}

In main i start 5 threads like this :

for (int x = 1; x <= 4; x++)
    {
        new ThreadComponent(x, this, featureMapCreator).init();
    }

Thread component initilize the connections and does the receive part

  Connector socketConnector = new Connector();
    socketConnector.establishConnection(60000 + index); // index are from 1 to 4
  while (true)
     {
            doLogicGetData(socketConnector);

     }
 In doLogicGetData the receive is called
 byte[] z = socketConnector.Receive(index);

I also tried to use a TcpClient with network stream but the result is the same.

What i am looking for is a Stream/Socket that is good for communication on local machine in a multithreading environment that reads all data in order. At the moment this implementation is not giving the all data all the time.

Answer 1

I suggest that you assign a unique port number for the six (6) process threads. This should prevent "different threads are interfering when receiving data.".

Also, you may not receive all transmitted data in a single get. Data transmissions can be split up into multiple packets that may require multiple gets to complete a single stream of transmitted data. Each data transmission should have some way to mark the end of the transmission, so that the get loop can tell when to stop getting the data to process the transmission.