Search code examples
apache-kafkakafka-consumer-apikafka-producer-apikafka-topic

Kafka: auto create topic with options

I'm using auto-creation of topics in Kafka. Some topics require compaction, some not.

Is it possible to specify custom options when auto-creating a topic; or do I need to test for existence and create it 'manually' with the desired options before I consume or produce on it?

Best regards

Solution

  • Enabling auto creation of topics allow you to write to a topic that does not exists. The only thing that you can define in your KafkaProducer is the name of the topic.

    All configurations for that topic will be set according to your default settings for topics, also called Server Default Property.

    In the documentation on the Topic Configuration you will find the corresponding configuration names, for example the Server Default Property for topic configcleanup.policy is broker config log.cleanup.policy.

    In case you need full control of checking for the existence of a topic and create it with specific configuration programatically, you can make use of the AdminClient.