I've got a local Kafka running using the following docker-compose.yml
file:
version: '2'
services:
zookeeper:
image: "confluentinc/cp-zookeeper:5.0.1"
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
kafka:
image: "confluentinc/cp-enterprise-kafka:5.0.1"
ports:
- '9092:9092'
depends_on:
- zookeeper
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
KAFKA_METRIC_REPORTERS: io.confluent.metrics.reporter.ConfluentMetricsReporter
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 100
Trying to run a basic create topic using kafka-client 2.1.0 in Scala:
val props = new Properties()
props.setProperty(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092")
val adminClient: AdminClient = AdminClient.create(props)
val newTopic = new NewTopic("test", 1, 1.toShort)
val topicsF = adminClient.createTopics(List(newTopic).asJavaCollection)
val result = topicsF.all().get()
but after some time I get:
org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
I can create a topic using the command line:
kafka-topics --create \
--zookeeper localhost:2181 \
--replication-factor 1 \
--partitions 1 \
--topic test
Created topic "test".
kafka AdminClient API Timed out waiting for node assignment describes a similar problem using Java but the comment suggests that a system restart fixed the issue which is not the case on my side.
If you're running Kafka in Docker (or similar) you need to configure the listeners correctly. This article describes it in detail.
Here's an example of a Docker Compose that you can use to access Kafka from your host machine.
Disclaimer: I wrote the article :)