Netty: Weird IndexOutOfBoundsException: readerIndex + length exceeds writerIndex

I am currently sending different packets through netty and I'm very often getting exceptions like these when receiving them:

java.lang.IndexOutOfBoundsException: readerIndex(39) + length(32) exceeds writerIndex(64): UnpooledUnsafeDirectByteBuf(ridx: 39, widx: 64, cap: 2048)
at io.netty.buffer.AbstractByteBuf.checkReadableBytes(AbstractByteBuf.java:1166)
at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:655) 
at <snip>.PacketDataHolder.readString(PacketDataHolder.java:79)

...... or

java.lang.IndexOutOfBoundsException: readerIndex(0) + length(1) exceeds writerIndex(0): UnpooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap: 2048)
at io.netty.buffer.AbstractByteBuf.checkReadableBytes(AbstractByteBuf.java:1166)
at io.netty.buffer.AbstractByteBuf.readByte(AbstractByteBuf.java:570) ......

Here's how I send packets:

public void sendPacket(Packet packet) {
    PacketDataHolder packetDataHolder = new PacketDataHolder(); //I'm creating a packetDataHolder, code is below
    packetDataHolder.writeHeader(packet); //Then i'm adding the packet ID as a header
    packet.writeData(packetDataHolder); //Then I'm writing the data of the packet (String for example)
    socketChannel.writeAndFlush(packetDataHolder.getByteBuf()); //Then I'm sending the packet
}

And how I receive and read them:

protected void channelRead0(ChannelHandlerContext ctx, ByteBuf byteBuf) throws Exception {
    PacketDataHolder packetDataHolder = new PacketDataHolder(byteBuf); //First I'm creating a PacketDataHolder containing the bytebuf
    Packet packet = PacketParser.parsePacket(packetDataHolder); //Then I'm parsing the header I wrote (The packet id)

Here's related code:

(PacketDataHolder class) public synchronized void writeHeader(Packet packet) { this.writeByte(packet.getId()); }

public synchronized byte readHeader() {
    return this.readByte();
}
public synchronized void writeByte(int value) {
    this.byteBuf.writeByte(value);
}

public synchronized byte readByte() {
    return this.byteBuf.readByte();
}
public synchronized void writeString(String value) {
    if(value == null)
        value = "An error occurred while displaying this message";
    byte[] bytes = value.getBytes(StandardCharsets.UTF_8);
    if (bytes.length > 32767)
        throw new EncoderException("String too big (was " + value.length()
                + " bytes encoded, max 32767)");
    else {
        writeVarInt(bytes.length);
        byteBuf.writeBytes(bytes);
    }
}

public synchronized String readString() {
    int length = readVarInt();
    if (length < 0)
        throw new DecoderException(
                "The received encoded string buffer length is less than zero! Weird string!");
    return new String(byteBuf.readBytes(length).array(),
            StandardCharsets.UTF_8);
}

The first error I described always happens at the writeString method, so I think in any way it's related to that. I think it's mostly when very long strings are sent, but I'm not sure.

If you need more example code feel free to ask! I hope you can help me, Kombustor

Solution

Your code assumes that your communication channel is handling framing. That is, it expects that when you send 1024 bytes, then the receive handler will receive a bytebuf of exactly 1024 bytes. As far as I can tell, your code is writing directly to TCP which is a stream based protocol and does not work this way. If you write 1024 bytes, the peer will receive all 1024 bytes, but the bytes may arrive in pieces - for example two calls channelRead0 (the first with 300 and the second with 724).

Fortunately, Netty includes several out-of-the-box framing codecs to handle this for you. See the example at http://netty.io/4.0/xref/io/netty/example/worldclock/WorldClockServerInitializer.html

It shows how to configure a server channel with the ProtobufVarint32FrameDecoder (which ensures the next handler in the pipeline always receives exactly one frame of bytes at a time) and the ProtobufVarint32LengthFieldPrepender (which takes care of adding the frame length header to all outgoing messages)

I recommend you change your code to insert these (or similar framing codecs) into your pipeline. Then modify your application code to not call readVarInt and writeVarInt since the codec will take care of this for you. The size of the received ByteBuf will always match the size of the sent ByteBuf because the decoder took care of aggregating any fragments.