KafkaJSBrokerNotFound: Broker -1 not found in the cached metadata

Question

trying to create topic with kafkajs in node js. using docker containers for Kafka and Zookeeper. getting the above error. what could be the reason? unfortunetly, googling it didn't help. i'm using windows 10 so working with WSL. Thanks

const {Kafka} = require("kafkajs")

run();
async function run(){
    try
    {
         const kafka = new Kafka({
              "clientId": "myapp",
              "brokers" :["localhost:9092"]
         })

        const admin = kafka.admin();
        console.log("Connecting.....")
        await admin.connect()
        console.log("Connected!")
        //A-M, N-Z
        await admin.createTopics({
            "topics": [{
                "topic" : "Users"
            }]
        })
        console.log("Created Successfully!")
        await admin.disconnect();
    }
    catch(ex)
    {
        console.error(`Something bad happened ${ex}`)
    }
    finally{
        process.exit(0);
    }


}

output is:

Connecting.....
Connected!
Something bad happened KafkaJSBrokerNotFound: Broker -1 not found in the cached metadata

docker-compose.yml:

services:
  zookeeper:
    image: zookeeper
    hostname: zookeeper
    ports:
      - 2181:2181

  kafka:
    restart: always
    image: ches/kafka
    hostname: localhost
    ports:
      - "9092:9092"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: 109.253.189.218
      ZOOKEEPER_IP: zookeeper
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      ALLOW_PLAINTEXT_LISTENER: 1
    
    depends_on:
      - zookeeper

DEBUG level logs: ....

Connected!
{"level":"DEBUG","timestamp":"2023-03-30T11:00:02.355Z","logger":"kafkajs","message":"[Connection] Request Metadata(key: 3, version: 2)","broker":"localhost:9092","clientId":"myapp","correlationId":3,"expectResponse":true,"size":23}
{"level":"DEBUG","timestamp":"2023-03-30T11:00:02.357Z","logger":"kafkajs","message":"[Connection] Response Metadata(key: 3, version: 2)","broker":"localhost:9092","clientId":"myapp","correlationId":3,"size":40,"data":{"brokers":[],"clusterId":"FsTkMSOPTlaLiIc5Qn65Fg","controllerId":-1,"topicMetadata":[]}}
{"level":"DEBUG","timestamp":"2023-03-30T11:00:02.357Z","logger":"kafkajs","message":"[Connection] Request Metadata(key: 3, version: 2)","broker":"localhost:9092","clientId":"myapp","correlationId":4,"expectResponse":true,"size":23}
{"level":"DEBUG","timestamp":"2023-03-30T11:00:02.359Z","logger":"kafkajs","message":"[Connection] Response Metadata(key: 3, version: 2)","broker":"localhost:9092","clientId":"myapp","correlationId":4,"size":40,"data":{"brokers":[],"clusterId":"FsTkMSOPTlaLiIc5Qn65Fg","controllerId":-1,"topicMetadata":[]}}
Something bad happened KafkaJSBrokerNotFound: Broker -1 not found in the cached metadata

Answer 1

Can you try using a maintained version of kafka (source):

# fill free to add more config
version: '2'

networks:
  app-tier:
    driver: bridge

services:
  zookeeper:
    image: 'bitnami/zookeeper:latest'
    networks:
      - app-tier
  kafka:
    image: 'bitnami/kafka:latest'
    networks:
      - app-tier
  myapp:
    image: 'YOUR_APPLICATION_IMAGE'
    networks:
      - app-tier