Is it possible to create kafka topics without additional service in docker-compose?

Question

I want to create and configure kafka topics on startup in docker-compose. I can do it with help of additions service (see init-kafka).

Is it possible to create kafka topics without additional service? I was trying to use command: kafka-topics.sh --create --bootstrap-server kafka:9092 --topic tasks_topic --partitions 3 in kafka service, but didn't succeed

This code is working for me:

version: "3.8"
    
services:
    zookeeper:
        image: bitnami/zookeeper:latest
        ports:
            - 2181:2181
        environment:
            - ALLOW_ANONYMOUS_LOGIN=yes

    kafka:
        image: bitnami/kafka:latest
        ports:
            - 9092:9092
            - 9093:9093
        environment:
            - KAFKA_BROKER_ID=1
            - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,CLIENT://:9093
            - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092,CLIENT://localhost:9093
            - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
            - ALLOW_PLAINTEXT_LISTENER=yes
            - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,PLAINTEXT:PLAINTEXT
            - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=CLIENT
        depends_on:
            - zookeeper
    
    init-kafka:
        image: bitnami/kafka:latest
        command: kafka-topics.sh   --create   --bootstrap-server kafka:9092 --topic tasks_topic --partitions 3

        depends_on:
            - kafka

UPDATE: I was trying this code (Added KAFKA_CREATE_TOPICS) but it didn't help

kafka:
        image: bitnami/kafka:latest
        ports:
            - 9092:9092
            - 9093:9093
        environment:
            - KAFKA_BROKER_ID=1
            - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,CLIENT://:9093
            - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092,CLIENT://localhost:9093
            - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
            - ALLOW_PLAINTEXT_LISTENER=yes
            - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,PLAINTEXT:PLAINTEXT
            - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=CLIENT
            - KAFKA_CREATE_TOPICS="my_tasks_topic1:5:1"
        depends_on:
            - zookeeper

Answer 1

There are multiple ways to do it. But for sure you can create through environment

kafka:
    image: bitnami/kafka:latest
    ports:
        - 9092:9092
        - 9093:9093
    environment:
        - KAFKA_BROKER_ID: 1
        - KAFKA_CFG_LISTENERS: PLAINTEXT://:9092,CLIENT://:9093
        - KAFKA_CFG_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,CLIENT://localhost:9093
        - KAFKA_CFG_ZOOKEEPER_CONNECT: zookeeper:2181
        - ALLOW_PLAINTEXT_LISTENER: yes
        - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CLIENT:PLAINTEXT,PLAINTEXT:PLAINTEXT
        - KAFKA_CFG_INTER_BROKER_LISTENER_NAME: CLIENT
        - KAFKA_CREATE_TOPICS: "tasks_topic1:3:1,another-topic:5:1"
    depends_on:
        - zookeeper

KAFKA_CREATE_TOPICS is environment variable will automatically create Kafka topics when the container starts. It uses the format topic_name:partitions:replication_factor.