Spring Boot and Kafka upgrade broke my tests

Question

I have updated the libraries of one project, in particular, I've updated Spring Boot, from version 2.2.6 to 2.3.2.

As mentioned in the migration documentation, this would also mean a change in my Kafka dependencies, going up to Spring Kafka and Kafka 2.5

However, since this change, some of my tests 'randomly' fail, as in, when I execute all of my tests, I encounter this failure in some of them (but not always in the same tests):

java.lang.IllegalStateException: More than one record for topic found

Coming from this particular line:

KafkaTestUtils.getSingleRecord(consumer, topicConfiguration.getTopic(event))

There is no mention in the documentation as to why this is happening, and no other log error message throwing light as to why this is happening. As I haven't changed the logic of my application, am I missing something with the migration?

If I try something like this:

KafkaTestUtils.getRecords(consumer)
    .records(topicConfiguration.getTopic(event)).map { it }[0]

To get only the first one, tests that worked previously would fail with an IndexOutOfBoundsException due to the index.

Answer 1

According to the test documentation of the Spring Kafka library, starting with version 2.5 they changed the way they configure the consumers. This is the exact quote:

Starting with version 2.5, the consumerProps method sets the ConsumerConfig.AUTO_OFFSET_RESET_CONFIG to earliest. This is because, in most cases, you want the consumer to consume any messages sent in a test case. The ConsumerConfig default is latest which means that messages already sent by a test, before the consumer starts, will not receive those records. To revert to the previous behavior, set the property to latest after calling the method.

It could be very likely the reason of the described test errors.