junitjunit4spring-kafkaspring-test
Not able to start embedded server for Kafka for Junit testing
I am trying to workout already implemented example from github for understanding of Springboot Kafka unit testing through Junit. I am new to testing. I am using this code Github When i try to run it as Junit then it gives me error mentioned below:
2021-06-16 18:14:15.907 INFO 9864 --- [ main] o.a.k.clients.producer.KafkaProducer : [Producer clientId=producer-1] Closing the Kafka producer with timeoutMillis = 0 ms.
2021-06-16 18:14:15.918 INFO 9864 --- [ -C-1] o.a.k.c.c.internals.ConsumerCoordinator : [Consumer clientId=consumer-sender-1, groupId=sender] Revoke previously assigned partitions UpdatedBrandEvent-0, UpdatedBrandEvent-1
2021-06-16 18:14:15.918 INFO 9864 --- [ -C-1] o.s.k.l.KafkaMessageListenerContainer : sender: partitions revoked: [UpdatedBrandEvent-0, UpdatedBrandEvent-1]
2021-06-16 18:14:15.918 INFO 9864 --- [ -C-1] o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-sender-1, groupId=sender] Member consumer-sender-1-95af1165-2ad8-4ea4-896b-28fbd9cef316 sending LeaveGroup request to coordinator localhost:60727 (id: 2147483647 rack: null) due to the consumer unsubscribed from all topics
2021-06-16 18:14:15.920 INFO 9864 --- [ -C-1] o.a.k.clients.consumer.KafkaConsumer : [Consumer clientId=consumer-sender-1, groupId=sender] Unsubscribed all topics or patterns and assigned partitions
2021-06-16 18:14:15.920 INFO 9864 --- [ -C-1] o.s.s.c.ThreadPoolTaskScheduler : Shutting down ExecutorService
2021-06-16 18:14:15.924 INFO 9864 --- [quest-handler-0] k.coordinator.group.GroupCoordinator : [GroupCoordinator 0]: Member[group.instance.id None, member.id consumer-sender-1-95af1165-2ad8-4ea4-896b-28fbd9cef316] in group sender has left, removing it from the group
2021-06-16 18:14:15.924 INFO 9864 --- [quest-handler-0] k.coordinator.group.GroupCoordinator : [GroupCoordinator 0]: Preparing to rebalance group sender in state PreparingRebalance with old generation 1 (__consumer_offsets-0) (reason: removing member consumer-sender-1-95af1165-2ad8-4ea4-896b-28fbd9cef316 on LeaveGroup)
2021-06-16 18:14:15.926 INFO 9864 --- [quest-handler-0] k.coordinator.group.GroupCoordinator : [GroupCoordinator 0]: Group sender with generation 2 is now empty (__consumer_offsets-0)
2021-06-16 18:14:15.936 INFO 9864 --- [ -C-1] essageListenerContainer$ListenerConsumer : sender: Consumer stopped
2021-06-16 18:14:15.964 INFO 9864 --- [ main] kafka.server.KafkaServer : [KafkaServer id=0] shutting down
2021-06-16 18:14:15.965 INFO 9864 --- [ main] kafka.server.KafkaServer : [KafkaServer id=0] Starting controlled shutdown
2021-06-16 18:14:15.982 INFO 9864 --- [er-event-thread] kafka.controller.KafkaController : [Controller id=0] Shutting down broker 0
2021-06-16 18:14:15.988 INFO 9864 --- [er-event-thread] state.change.logger : [Controller id=0 epoch=1] Sending UpdateMetadata request to brokers HashSet() for 0 partitions
2021-06-16 18:14:15.993 INFO 9864 --- [ main] kafka.server.KafkaServer : [KafkaServer id=0] Controlled shutdown succeeded
2021-06-16 18:14:16.002 INFO 9864 --- [ main] icationListener$ChangeEventProcessThread : [/config/changes-event-process-thread]: Shutting down
2021-06-16 18:14:16.003 INFO 9864 --- [-process-thread] icationListener$ChangeEventProcessThread : [/config/changes-event-process-thread]: Stopped
2021-06-16 18:14:16.003 INFO 9864 --- [ main] icationListener$ChangeEventProcessThread : [/config/changes-event-process-thread]: Shutdown completed
2021-06-16 18:14:16.003 INFO 9864 --- [ main] kafka.network.SocketServer : [SocketServer brokerId=0] Stopping socket server request processors
2021-06-16 18:14:16.012 INFO 9864 --- [ main] kafka.network.SocketServer : [SocketServer brokerId=0] Stopped socket server request processors
2021-06-16 18:14:16.013 INFO 9864 --- [ main] kafka.server.KafkaRequestHandlerPool : [data-plane Kafka Request Handler on Broker 0], shutting down
2021-06-16 18:14:16.016 INFO 9864 --- [ main] kafka.server.KafkaRequestHandlerPool : [data-plane Kafka Request Handler on Broker 0], shut down completely
2021-06-16 18:14:16.020 INFO 9864 --- [ main] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-AlterAcls]: Shutting down
2021-06-16 18:14:16.220 INFO 9864 --- [per-0-AlterAcls] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-AlterAcls]: Stopped
2021-06-16 18:14:16.220 INFO 9864 --- [ main] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-AlterAcls]: Shutdown completed
2021-06-16 18:14:16.221 INFO 9864 --- [ main] kafka.server.KafkaApis : [KafkaApi-0] Shutdown complete.
2021-06-16 18:14:16.222 INFO 9864 --- [ main] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-topic]: Shutting down
2021-06-16 18:14:16.309 INFO 9864 --- [ main] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-topic]: Shutdown completed
2021-06-16 18:14:16.309 INFO 9864 --- [nReaper-0-topic] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-topic]: Stopped
2021-06-16 18:14:16.312 INFO 9864 --- [ main] k.c.transaction.TransactionCoordinator : [TransactionCoordinator id=0] Shutting down.
2021-06-16 18:14:16.313 INFO 9864 --- [ main] k.c.transaction.ProducerIdManager : [ProducerId Manager 0]: Shutdown complete: last producerId assigned 0
2021-06-16 18:14:16.314 INFO 9864 --- [ main] k.c.transaction.TransactionStateManager : [Transaction State Manager 0]: Shutdown complete
2021-06-16 18:14:16.314 INFO 9864 --- [ main] k.c.t.TransactionMarkerChannelManager : [Transaction Marker Channel Manager 0]: Shutting down
2021-06-16 18:14:16.318 INFO 9864 --- [rSenderThread-0] k.c.t.TransactionMarkerChannelManager : [Transaction Marker Channel Manager 0]: Stopped
2021-06-16 18:14:16.318 INFO 9864 --- [ main] k.c.t.TransactionMarkerChannelManager : [Transaction Marker Channel Manager 0]: Shutdown completed
2021-06-16 18:14:16.319 INFO 9864 --- [ main] k.c.transaction.TransactionCoordinator : [TransactionCoordinator id=0] Shutdown complete.
2021-06-16 18:14:16.320 INFO 9864 --- [ main] k.coordinator.group.GroupCoordinator : [GroupCoordinator 0]: Shutting down.
2021-06-16 18:14:16.321 INFO 9864 --- [ main] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-Heartbeat]: Shutting down
2021-06-16 18:14:16.510 INFO 9864 --- [ main] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-Heartbeat]: Shutdown completed
2021-06-16 18:14:16.510 INFO 9864 --- [per-0-Heartbeat] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-Heartbeat]: Stopped
2021-06-16 18:14:16.511 INFO 9864 --- [ main] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-Rebalance]: Shutting down
2021-06-16 18:14:16.544 INFO 9864 --- [ main] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-Rebalance]: Shutdown completed
2021-06-16 18:14:16.544 INFO 9864 --- [per-0-Rebalance] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-Rebalance]: Stopped
2021-06-16 18:14:16.545 INFO 9864 --- [ main] k.coordinator.group.GroupCoordinator : [GroupCoordinator 0]: Shutdown complete.
2021-06-16 18:14:16.545 INFO 9864 --- [ main] kafka.server.ReplicaManager : [ReplicaManager broker=0] Shutting down
2021-06-16 18:14:16.546 INFO 9864 --- [ main] k.s.ReplicaManager$LogDirFailureHandler : [LogDirFailureHandler]: Shutting down
2021-06-16 18:14:16.546 INFO 9864 --- [rFailureHandler] k.s.ReplicaManager$LogDirFailureHandler : [LogDirFailureHandler]: Stopped
2021-06-16 18:14:16.546 INFO 9864 --- [ main] k.s.ReplicaManager$LogDirFailureHandler : [LogDirFailureHandler]: Shutdown completed
2021-06-16 18:14:16.546 INFO 9864 --- [ main] kafka.server.ReplicaFetcherManager : [ReplicaFetcherManager on broker 0] shutting down
2021-06-16 18:14:16.548 INFO 9864 --- [ main] kafka.server.ReplicaFetcherManager : [ReplicaFetcherManager on broker 0] shutdown completed
2021-06-16 18:14:16.548 INFO 9864 --- [ main] kafka.server.ReplicaAlterLogDirsManager : [ReplicaAlterLogDirsManager on broker 0] shutting down
2021-06-16 18:14:16.548 INFO 9864 --- [ main] kafka.server.ReplicaAlterLogDirsManager : [ReplicaAlterLogDirsManager on broker 0] shutdown completed
2021-06-16 18:14:16.548 INFO 9864 --- [ main] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-Fetch]: Shutting down
2021-06-16 18:14:16.618 INFO 9864 --- [ main] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-Fetch]: Shutdown completed
2021-06-16 18:14:16.618 INFO 9864 --- [nReaper-0-Fetch] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-Fetch]: Stopped
2021-06-16 18:14:16.618 INFO 9864 --- [ main] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-Produce]: Shutting down
2021-06-16 18:14:16.726 INFO 9864 --- [ main] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-Produce]: Shutdown completed
2021-06-16 18:14:16.726 INFO 9864 --- [eaper-0-Produce] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-Produce]: Stopped
2021-06-16 18:14:16.726 INFO 9864 --- [ main] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-DeleteRecords]: Shutting down
2021-06-16 18:14:16.744 INFO 9864 --- [ main] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-DeleteRecords]: Shutdown completed
2021-06-16 18:14:16.744 INFO 9864 --- [0-DeleteRecords] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-DeleteRecords]: Stopped
2021-06-16 18:14:16.744 INFO 9864 --- [ main] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-ElectLeader]: Shutting down
2021-06-16 18:14:16.945 INFO 9864 --- [r-0-ElectLeader] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-ElectLeader]: Stopped
2021-06-16 18:14:16.945 INFO 9864 --- [ main] perationPurgatory$ExpiredOperationReaper : [ExpirationReaper-0-ElectLeader]: Shutdown completed
2021-06-16 18:14:16.956 INFO 9864 --- [ main] kafka.server.ReplicaManager : [ReplicaManager broker=0] Shut down completely
2021-06-16 18:14:16.957 INFO 9864 --- [ main] kafka.log.LogManager : Shutting down.
2021-06-16 18:14:16.959 INFO 9864 --- [ main] kafka.log.LogCleaner : Shutting down the log cleaner.
2021-06-16 18:14:16.960 INFO 9864 --- [ main] kafka.log.LogCleaner : [kafka-log-cleaner-thread-0]: Shutting down
2021-06-16 18:14:16.961 INFO 9864 --- [leaner-thread-0] kafka.log.LogCleaner : [kafka-log-cleaner-thread-0]: Stopped
2021-06-16 18:14:16.961 INFO 9864 --- [ main] kafka.log.LogCleaner : [kafka-log-cleaner-thread-0]: Shutdown completed
2021-06-16 18:14:17.129 INFO 9864 --- [pool-8-thread-1] kafka.log.ProducerStateManager : [ProducerStateManager partition=__consumer_offsets-0] Writing producer snapshot at offset 2
2021-06-16 18:14:17.280 INFO 9864 --- [ main] kafka.log.LogManager : Shutdown complete.
2021-06-16 18:14:17.280 INFO 9864 --- [ main] rollerEventManager$ControllerEventThread : [ControllerEventThread controllerId=0] Shutting down
2021-06-16 18:14:17.281 INFO 9864 --- [ main] rollerEventManager$ControllerEventThread : [ControllerEventThread controllerId=0] Shutdown completed
2021-06-16 18:14:17.281 INFO 9864 --- [er-event-thread] rollerEventManager$ControllerEventThread : [ControllerEventThread controllerId=0] Stopped
2021-06-16 18:14:17.285 INFO 9864 --- [ main] k.controller.ZkPartitionStateMachine : [PartitionStateMachine controllerId=0] Stopped partition state machine
2021-06-16 18:14:17.286 INFO 9864 --- [ main] kafka.controller.ZkReplicaStateMachine : [ReplicaStateMachine controllerId=0] Stopped replica state machine
2021-06-16 18:14:17.287 INFO 9864 --- [ main] kafka.controller.RequestSendThread : [RequestSendThread controllerId=0] Shutting down
2021-06-16 18:14:17.287 INFO 9864 --- [r-0-send-thread] kafka.controller.RequestSendThread : [RequestSendThread controllerId=0] Stopped
2021-06-16 18:14:17.287 INFO 9864 --- [ main] kafka.controller.RequestSendThread : [RequestSendThread controllerId=0] Shutdown completed
2021-06-16 18:14:17.288 INFO 9864 --- [ main] kafka.controller.KafkaController : [Controller id=0] Resigned
2021-06-16 18:14:17.289 INFO 9864 --- [ main] kafka.zookeeper.ZooKeeperClient : [ZooKeeperClient Kafka server] Closing.
2021-06-16 18:14:17.395 INFO 9864 --- [ain-EventThread] org.apache.zookeeper.ClientCnxn : EventThread shut down for session: 0x1000683bf6a0000
2021-06-16 18:14:17.395 INFO 9864 --- [ main] org.apache.zookeeper.ZooKeeper : Session: 0x1000683bf6a0000 closed
2021-06-16 18:14:17.396 INFO 9864 --- [ main] kafka.zookeeper.ZooKeeperClient : [ZooKeeperClient Kafka server] Closed.
2021-06-16 18:14:17.396 INFO 9864 --- [ main] lientQuotaManager$ThrottledChannelReaper : [ThrottledChannelReaper-Fetch]: Shutting down
2021-06-16 18:14:17.447 INFO 9864 --- [ main] lientQuotaManager$ThrottledChannelReaper : [ThrottledChannelReaper-Fetch]: Shutdown completed
2021-06-16 18:14:17.447 INFO 9864 --- [nelReaper-Fetch] lientQuotaManager$ThrottledChannelReaper : [ThrottledChannelReaper-Fetch]: Stopped
2021-06-16 18:14:17.447 INFO 9864 --- [ main] lientQuotaManager$ThrottledChannelReaper : [ThrottledChannelReaper-Produce]: Shutting down
2021-06-16 18:14:18.447 INFO 9864 --- [lReaper-Produce] lientQuotaManager$ThrottledChannelReaper : [ThrottledChannelReaper-Produce]: Stopped
2021-06-16 18:14:18.447 INFO 9864 --- [ main] lientQuotaManager$ThrottledChannelReaper : [ThrottledChannelReaper-Produce]: Shutdown completed
2021-06-16 18:14:18.447 INFO 9864 --- [ main] lientQuotaManager$ThrottledChannelReaper : [ThrottledChannelReaper-Request]: Shutting down
2021-06-16 18:14:18.453 INFO 9864 --- [ main] lientQuotaManager$ThrottledChannelReaper : [ThrottledChannelReaper-Request]: Shutdown completed
2021-06-16 18:14:18.453 INFO 9864 --- [lReaper-Request] lientQuotaManager$ThrottledChannelReaper : [ThrottledChannelReaper-Request]: Stopped
2021-06-16 18:14:18.454 INFO 9864 --- [ main] kafka.network.SocketServer : [SocketServer brokerId=0] Shutting down socket server
2021-06-16 18:14:18.482 INFO 9864 --- [ main] kafka.network.SocketServer : [SocketServer brokerId=0] Shutdown completed
2021-06-16 18:14:18.485 INFO 9864 --- [ main] kafka.server.KafkaServer : [KafkaServer id=0] shut down completed
2021-06-16 18:14:18.509 INFO 9864 --- [nnectionExpirer] o.a.z.server.NIOServerCnxnFactory : ConnnectionExpirerThread interrupted
2021-06-16 18:14:18.509 INFO 9864 --- [electorThread-1] o.a.z.server.NIOServerCnxnFactory : selector thread exitted run method
2021-06-16 18:14:18.509 INFO 9864 --- [ad:/127.0.0.1:0] o.a.z.server.NIOServerCnxnFactory : accept thread exitted run method
2021-06-16 18:14:18.510 INFO 9864 --- [electorThread-0] o.a.z.server.NIOServerCnxnFactory : selector thread exitted run method
2021-06-16 18:14:18.510 INFO 9864 --- [ main] o.a.zookeeper.server.ZooKeeperServer : shutting down
2021-06-16 18:14:18.511 INFO 9864 --- [ main] o.a.zookeeper.server.SessionTrackerImpl : Shutting down
2021-06-16 18:14:18.511 INFO 9864 --- [ main] o.a.z.server.PrepRequestProcessor : Shutting down
2021-06-16 18:14:18.511 INFO 9864 --- [ main] o.a.z.server.SyncRequestProcessor : Shutting down
2021-06-16 18:14:18.511 INFO 9864 --- [0 cport:60719):] o.a.z.server.PrepRequestProcessor : PrepRequestProcessor exited loop!
2021-06-16 18:14:18.511 INFO 9864 --- [ SyncThread:0] o.a.z.server.SyncRequestProcessor : SyncRequestProcessor exited!
2021-06-16 18:14:18.511 INFO 9864 --- [ main] o.a.z.server.FinalRequestProcessor : shutdown of request processor complete
2021-06-16 18:14:19.207 INFO 9864 --- [ SessionTracker] o.a.zookeeper.server.SessionTrackerImpl : SessionTrackerImpl exited loop!
Please help me here as i am running this on my local.Do i need to modify any other configuration files in bin for that.
Solution
That project is using an old, unsupported, version of Boot. I had problems with Zookeeper with that version on my Mac.
You are also missing spring: in the application.yml.
Here is what I did to get the test to pass.
Change the pom to use a supported Boot:
...
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.3.12.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
...
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka-test</artifactId>
<scope>test</scope>
</dependency>
...
Fix the Yaml (main)
spring:
kafka:
bootstrap-servers: localhost:9092
Fix the Yaml (test)
spring:
kafka:
bootstrap-servers: ${spring.embedded.kafka.brokers}
Fix the KafkaProducerConfiguration
@Value("${spring.kafka.bootstrap-servers}")
private String bootstrapServers;
If you used a boot-aware IDE (such as the Spring Tool Suite or IntelliJ you would have seen warnings about the bad YAML.
- Keep numbers which appear in both columns, in J lang
- Differentiation in J
- How to reshape an array with an arbitrary size in one dimension?
- Why is Insert (fold) right associative
- Write 4 : 'x&{.&.;: y' tacitly
- Alignment issue when printing formatted prime numbers in J language
- How can I define a verb in J that applies a different verb alternately to each atom in a list?
- How to get user input in the J programming language
- How to unbox a list of boxed lists of differing lengths in J?
- How can I fix 'noun result was required' error in J?
- In j, how can I define a verb locally in one scope and pass it to a defined adverb?
- Convert boxed array to normal array?
- Read column of CSV file as array
- Replace atom in array of strings
- What does the dyad `=` do to boxed strings?
- Index of minimum element using J
- How can I take the outer product of string vectors in J?
- Building an array of verbs in J
- Reading in multidigit command line parameter
- Amend with bond to new data shows unexpected behaviour
- How to turn a table or matrix into a (flat) list in J
- How to run dissect in J?
- How to define selection using index function in J
- How to exit the J console?
- Find 4-neighbors using J
- Writing custom verbs in J
- How do I negate a selector in J lang?
- How to use arbitrary selector in interchange in J lang?
- different result once square root is added inside tacit
- Sum of arrays with repeated indices