netty packet infinite loop occur

Question

project setting netty 4.1.65final + kotlin 1.5 + spring boot 2.5

The server deployment environment is using kubernets and docker baseimage is using alpine-adopt-openjdk-11.0.11.

In the process of restarting the Netty server, tcp packets enter infinitely.

It is occurring through the same netty channel, and when I checked the port in remoteAddress on the channel and connected to the place where the server was floating and searched "netstat-an", I could not find the port number.

I tried to close the channel when the same packet comes in, but the channel doesn't close.

I think there's an infinite loop on a particular channel in netty, so why is that?

below my server log

{"log_level":"INFO","time":"2021-07-14T03:22:31.376Z","log_type":"TCP_LOG","msg":"@@ duplicate packet. channel close"}
{"log_level":"INFO","time":"2021-07-14T03:22:31.376Z","log_type":"TCP_LOG","data_type":"PACKET","byte_array":"AAAA0024450560000000455400002026403695A46000FFFFCBAC","remote_ip":"/10.240.2.129:37508","channel_id":"a437de98"}
{"log_level":"INFO","time":"2021-07-14T03:22:31.388Z","log_type":"TCP_LOG","transfer_type":"SEND","data_type":"HEX","byte_array":"aaaa000c458560000000115a"}
{"log_level":"INFO","time":"2021-07-14T03:22:31.388Z","log_type":"TCP_LOG","msg":"@@ duplicate packet. channel close"}
{"log_level":"INFO","time":"2021-07-14T03:22:31.388Z","log_type":"TCP_LOG","data_type":"PACKET","byte_array":"AAAA0024450560000000455400002026403695A46000FFFFCBAC","remote_ip":"/10.240.2.129:37508","channel_id":"a437de98"}
{"log_level":"INFO","time":"2021-07-14T03:22:31.393Z","log_type":"TCP_LOG","transfer_type":"SEND","data_type":"HEX","byte_array":"aaaa000c458560000000115a"}

additional

Replacing Decoder with LengthFieldBasedFrameDecoder did not cause any problems.

When ReplayingDecoder completes receiving the packet, it should empty the buf, but I suspect it was caused by not emptying it.

Below is the Replaying Decoder I wrote.

enum class PacketDecoderState {
    READ_HEADER, READ_WHOLE
}

@ExperimentalUnsignedTypes
class PacketReceiverHandler : ReplayingDecoder<PacketDecoderState>(PacketDecoderState.READ_HEADER) {
    private var length: Int = 0
    override fun decode(ctx: ChannelHandlerContext, buf: ByteBuf, out: MutableList<Any>) {
        when (state()) {
            PacketDecoderState.READ_HEADER -> {
                println("READ HEAD :: $length | ${ctx.channel().remoteAddress()} | ${ctx.channel().id()} | $this | ${Thread.currentThread().name}")
                val headerPacket = buf.readBytes(HeaderSize.TOTAL_HEAD_SIZE)

                val startCode = ByteBufUtil.getBytes(headerPacket.readBytes(HeaderSize.START_CODE.length))
                startCode.takeIf { !it.contentEquals(START_CODE) }?.let {
                    ctx.channel().close()
                    val array = ByteArray(buf.readableBytes())
                    buf.getBytes(buf.readerIndex(), array)
                    throw TcpPacketValidationCheckException(TcpError(errorCode = ErrorCode.P005, byteArray = array))
                }
                length = headerPacket.readBytes(HeaderSize.PACKET_LENGTH.length).getUnsignedShort(0)
                checkpoint(PacketDecoderState.READ_WHOLE)

                buf.resetReaderIndex()
            }

            PacketDecoderState.READ_WHOLE -> {
                println("READ_WHOLE :: $length | ${ctx.channel().remoteAddress()} | ${ctx.channel().id()} | $this | ${Thread.currentThread().name} ")

                val bodyPacket = buf.readBytes(length)
                checkpoint(PacketDecoderState.READ_HEADER)
                out.add(bodyPacket)
            }
            else -> throw ProtocolsException(AppError(ErrorCode.P000))
        }
    }

}

Answer 1

When duplicate packets come in, an infinite loop occurs while reading header information and resetting the read index.

For normal packets

aaaa00244505f4000000754e672f4a78435562716b6d71755a636f3253496a673d3d51ea

The first 10 bytes are the header information.

But when the server restarts and packets pile up in the buffer and come in at once. aaaa00244505f4000000754e672f4a78435562716b6d71755a636f3253496a673d3d51eaaaaa00244505f4000000754e672f4a78435562716b6d71755a636f3253496a673d3d51ea

Two packets are piled up in ByteBuf one after that.

Operating inside the ReplayingDecoder

@Override
protected void callDecode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) {
replayable.setCumulation(in);
try {
while (in.isReadable()) {
int oldReaderIndex = checkpoint = in.readerIndex();

while (in.isReadable()) because there's still data left in ByteBuf, so it keeps running.

Read the header information in the code I wrote and buf.ResetReaderIndex() initializes ByteBuf's read index to zero, which causes infinite loops.