I'm implementing a client server communication via socketChannel. During load tests with larger objets I run into problems on the client side. So I've implemented a test program to verify my problem an to illustrate it here.
First a small explaination: I don't know on client side the size of the object that will be send through the socket. So I split my sending into two parts: 1.: serializition of the object into a byteBuffer on server side (in the example illustrated with the byte-array. 2.: sending the soze of the object through the socket 3.: sending the object
On client side, I first read the object size in a 4-byte ByteBuffer. As second I create a new ByteBuffer with the readed size and then read the data from the socketchannel into the Buffer.
If you have a look a the code, than you can see (In the client class), that my expectation is, that the socketChannel.read method will return the same byte count as the readed object size before.
But after increasing the sendet byte array, there will bee a lot of missmatches and zero sizes. Why this happends?? the socketchannel is nonblocking. so it should be able to read everything in the configured byte buffer. the size of the byte buffer is big enough. so why are there sometimes bytes missing?
Thank you very much! Here is the example code:
Main Class
package socketChannelMaxTest;
import java.io.IOException;
public class MaxTest {
public static void main(String[] args) throws IOException {
Sender.startServer();
new Client();
}
}
Server Sender
package socketChannelMaxTest;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
public class Sender {
private SocketChannel sc;
public Sender(SocketChannel sc) {
this.sc = sc;
startThreads();
}
public static void startServer() throws IOException {
ServerSocketChannel ssc = ServerSocketChannel.open();
ServerSocketChannel socket = ssc.bind(new InetSocketAddress(9999));
new Thread(() -> {
System.out.println("Server: Listening");
SocketChannel sc;
try {
sc = socket.accept();
sc.configureBlocking(true);
new Sender(sc);
} catch (IOException e) {
e.printStackTrace();
}
}).start();
}
private void startThreads() {
new Thread(() -> {
System.out.println("Sender: start sending");
ByteBuffer headerBuffer = ByteBuffer.allocateDirect(4);
int maxBufferSize = 10*1024;
for (int i = 1; i < maxBufferSize; i++) {
byte[] randomByteArray = new byte[i];
ByteBuffer dataBuffer = ByteBuffer.wrap(randomByteArray);
int objectSize = randomByteArray.length;
headerBuffer.putInt(objectSize);
headerBuffer.flip();
try {
sc.write(headerBuffer);
System.out.println("Sender: " + objectSize + " " + sc.write(dataBuffer));
} catch (IOException e) {
e.printStackTrace();
}
headerBuffer.compact();
}
System.out.println("Sender: finished");
}, "Receiver Thread").start();
}
}
Client Receiver
package socketChannelMaxTest;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SocketChannel;
public class Client {
public Client() throws IOException {
startThreads();
}
private void startThreads() throws IOException {
System.out.println("Client: start client");
SocketChannel socketChannel = SocketChannel.open(new InetSocketAddress("localhost", 9999));
socketChannel.configureBlocking(true);
System.out.println("Client: connected");
new Thread(() -> {
System.out.println("Client: start listening");
ByteBuffer headerBuffer = ByteBuffer.allocate(4);
int readedObjectSize = 0;
while (socketChannel.isConnected()) {
try {
int read = socketChannel.read(headerBuffer);
headerBuffer.flip();
readedObjectSize = headerBuffer.getInt();
headerBuffer.compact();
ByteBuffer dataBuffer = ByteBuffer.allocateDirect(readedObjectSize);
int readedDataBufferSize = socketChannel.read(dataBuffer);
// should be 0
int remainginBytes = dataBuffer.remaining();
dataBuffer.flip();
System.out.println("Client:" + readedObjectSize + " " + readedDataBufferSize + " " + remainginBytes);
if (readedObjectSize != readedDataBufferSize)
System.out.println("Missmatch");
} catch (Exception e1) {
e1.printStackTrace();
}
}
}, "Receiver Thread").start();
}
}
There is a problem with the fact that you assume your remaining data will always be 0 at any time
// should be 0
int remainginBytes = dataBuffer.remaining();
You can not be sure that this will be the case at any time. If the data gets too big, you will need to read again from your SocketChannel to retrieve the remaining information.
You should have a loop that reads from the socketChannel until there is no remaining bytes
int read = socketChannel.read(headerBuffer);
headerBuffer.flip();
readedObjectSize = headerBuffer.getInt();
headerBuffer.compact();
int remainginBytes = -1;
int readedDataBufferSize = 0;
ByteBuffer dataBuffer = ByteBuffer.allocateDirect(readedObjectSize);
while(remainginBytes != 0){
readedDataBufferSize = socketChannel.read(dataBuffer);
remainginBytes = dataBuffer.remaining();
}
dataBuffer.flip();
System.out.println("Client:" + readedObjectSize + " " + readedDataBufferSize + " " + remainginBytes);
if (readedObjectSize != readedDataBufferSize)
System.out.println("Missmatch");
Remember that configuring a socket to non-blocking mode will not ensure you that it will not return before all the data is retrieved
Feel free to comment if this still does not work for you !