I am try to setup Kafka cluster with provectuslabs-ui in docker:
name: 'test-kafka'
services:
kafka-1:
image: 'bitnami/kafka:latest'
container_name: 'kafka-1'
ports:
- '9092:9092'
- '29092:29092'
environment:
# KRaft mode settings
KAFKA_ENABLE_KRAFT: 'true'
KAFKA_CFG_NODE_ID: 1
KAFKA_BROKER_ID: 1
KAFKA_CFG_CLUSTER:ID: 'abcdefghijklmnopqrstuv'
KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@kafka-1:9092,2@kafka-2:9093,3@kafka-3:9094
KAFKA_CFG_PROCESS_ROLES: controller,broker
# Listeners settings
KAFKA_CFG_LISTENERS: BROKER://:9092,CONTROLLER://0.0.0.0:29092
KAFKA_CFG_ADVERTISED_LISTENERS: BROKER://kafka-1:9092,CONTROLLER://localhost:29092
KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,BROKER:PLAINTEXT
KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER
KAFKA_CFG_INTER_BROKER_LISTENER_NAME: BROKER
# Clustering
KAFKA_CFG_OFFSETS_TOPIC_REPLICATION_FACTOR: 3
KAFKA_CFG_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 3
KAFKA_CFG_TRANSACTION_STATE_LOG_MIN_ISR: 2
kafka-2:
image: 'bitnami/kafka:latest'
container_name: 'kafka-2'
ports:
- '9093:9093'
- '29093:29093'
environment:
# KRaft mode settings
KAFKA_ENABLE_KRAFT: 'true'
KAFKA_CFG_NODE_ID: 2
KAFKA_BROKER_ID: 2
KAFKA_CFG_CLUSTER:ID: 'abcdefghijklmnopqrstuv'
KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@kafka-1:9092,2@kafka-2:9093,3@kafka-3:9094
KAFKA_CFG_PROCESS_ROLES: controller,broker
# Listeners settings
KAFKA_CFG_LISTENERS: BROKER://:9093,CONTROLLER://0.0.0.0:29093
KAFKA_CFG_ADVERTISED_LISTENERS: BROKER://kafka-2:9093,CONTROLLER://localhost:29093
KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,BROKER:PLAINTEXT
KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER
KAFKA_CFG_INTER_BROKER_LISTENER_NAME: BROKER
# Clustering
KAFKA_CFG_OFFSETS_TOPIC_REPLICATION_FACTOR: 3
KAFKA_CFG_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 3
KAFKA_CFG_TRANSACTION_STATE_LOG_MIN_ISR: 2
kafka-3:
image: 'bitnami/kafka:latest'
container_name: 'kafka-3'
ports:
- '9094:9094'
- '29094:29094'
environment:
# KRaft mode settings
KAFKA_ENABLE_KRAFT: 'true'
KAFKA_CFG_NODE_ID: 3
KAFKA_BROKER_ID: 3
KAFKA_CFG_CLUSTER:ID: 'abcdefghijklmnopqrstuv'
KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@kafka-1:9092,2@kafka-2:9093,3@kafka-3:9094
KAFKA_CFG_PROCESS_ROLES: controller,broker
# Listeners settings
KAFKA_CFG_LISTENERS: BROKER://:9094,CONTROLLER://0.0.0.0:29094
KAFKA_CFG_ADVERTISED_LISTENERS: BROKER://kafka-3:9094,CONTROLLER://localhost:29094
KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,BROKER:PLAINTEXT
KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER
KAFKA_CFG_INTER_BROKER_LISTENER_NAME: BROKER
# Clustering
KAFKA_CFG_OFFSETS_TOPIC_REPLICATION_FACTOR: 3
KAFKA_CFG_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 3
KAFKA_CFG_TRANSACTION_STATE_LOG_MIN_ISR: 2
kafka-ui:
image: 'provectuslabs/kafka-ui:latest'
container_name: 'kafka-ui'
ports:
- '1354:8080'
depends_on:
- kafka-1
- kafka-2
- kafka-3
environment:
KAFKA_CLUSTERS_0_NAME: test-local
KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka-1:9092,kafka-2:9093,kafka-3:9094
When I up my services I get errors:
UI error:
2025-01-15 19:02:33 2025-01-15 16:02:33,796 INFO [kafka-admin-client-thread | kafka-ui-admin-1736956953-1] o.a.k.c.NetworkClient: [AdminClient clientId=kafka-ui-admin-1736956953-1] Node -1 disconnected.
2025-01-15 19:02:33 2025-01-15 16:02:33,800 WARN [kafka-admin-client-thread | kafka-ui-admin-1736956953-1] o.a.k.c.NetworkClient: [AdminClient clientId=kafka-ui-admin-1736956953-1] Connection to node -1 (kafka-1/172.18.0.3:9092) could not be established. Broker may not be available.
2025-01-15 19:02:33 2025-01-15 16:02:33,807 INFO [kafka-admin-client-thread | kafka-ui-admin-1736956953-1] o.a.k.c.NetworkClient: [AdminClient clientId=kafka-ui-admin-1736956953-1] Node -2 disconnected.
2025-01-15 19:02:33 2025-01-15 16:02:33,808 WARN [kafka-admin-client-thread | kafka-ui-admin-1736956953-1] o.a.k.c.NetworkClient: [AdminClient clientId=kafka-ui-admin-1736956953-1] Connection to node -2 (kafka-2/172.18.0.4:9093) could not be established. Broker may not be available.
2025-01-15 19:02:33 2025-01-15 16:02:33,812 INFO [kafka-admin-client-thread | kafka-ui-admin-1736956953-1] o.a.k.c.NetworkClient: [AdminClient clientId=kafka-ui-admin-1736956953-1] Node -3 disconnected.
2025-01-15 19:02:33 2025-01-15 16:02:33,813 WARN [kafka-admin-client-thread | kafka-ui-admin-1736956953-1] o.a.k.c.NetworkClient: [AdminClient clientId=kafka-ui-admin-1736956953-1] Connection to node -3 (kafka-3/172.18.0.2:9094) could not be established. Broker may not be available.
Node error (example for kafka-1 node):
2025-01-15 19:03:18 [2025-01-15 16:03:18,188] INFO [RaftManager id=1] Node 2 disconnected. (org.apache.kafka.clients.NetworkClient)
2025-01-15 19:03:18 [2025-01-15 16:03:18,188] WARN [RaftManager id=1] Connection to node 2 (kafka-2/172.18.0.4:9093) could not be established. Node may not be available. (org.apache.kafka.clients.NetworkClient)
2025-01-15 19:03:18 [2025-01-15 16:03:18,188] INFO [RaftManager id=1] Node 3 disconnected. (org.apache.kafka.clients.NetworkClient)
2025-01-15 19:03:18 [2025-01-15 16:03:18,188] WARN [RaftManager id=1] Connection to node 3 (kafka-3/172.18.0.2:9094) could not be established. Node may not be available. (org.apache.kafka.clients.NetworkClient)
There's a bunch of ready-to-use compose configs available instead: https://github.com/kafbat/kafka-ui/tree/main/documentation/compose Feel free to try out one of them.
In your particular case, there should be a plaintext advertised listener rather than a broker one:
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka0:29092,PLAINTEXT_HOST://localhost:9092
If you have any further questions, feel free to join us in discussions on github or in our discord server (the link is available in repo's readme).