I am Getting an error: org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 3 larger than available brokers: 1

Question

I am getting my this error as I am running by Kafka Application in Spring Boot. I want to create topic via running my application. This is my config class:

@Configuration
public class KafkaConfig {

    @Bean
    NewTopic createTopic() {
        return TopicBuilder.name("product-created-events-topic")
                .partitions(3)
                .replicas(3)
                .configs(Map.of("min.insync.replicas", "2"))
                .build();
    }
}

My application.properties:

spring.application.name=Product-Microservice
server.port=0
server.kafka.producer.bootstrap-servers=localhost:9092, localhost:9094
spring-kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring-kafka.producer.value-serializer=org.springframework.kafka.support.serializer.JsonSerializer

This is the Error shown in my kafka server:

[2024-06-17 09:45:17,355] INFO [Admin Manager on Broker 0]: Error processing create topic request CreatableTopic(name='product-created-events-topic', numPartitions=3, replicationFactor=3, assignments=[], configs=[CreateableTopicConfig(name='min.insync.replicas', value='2')]) (kafka.server.ZkAdminManager)
org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 3 larger than available brokers: 1.

When I changed replicas(3) to replicas(1), it is working and my topic is created:

@Configuration
public class KafkaConfig {

    @Bean
    NewTopic createTopic() {
        return TopicBuilder.name("product-created-events-topic")
                .partitions(3)
                .replicas(1) //Changed from 3 -> 1
                .configs(Map.of("min.insync.replicas", "2"))
                .build();
    }
}

after running this my process was created but with only 1 replica

Topic: product-created-events-topic     TopicId: ---- PartitionCount: 3       ReplicationFactor: 1    Configs: min.insync.replicas=2
        Topic: product-created-events-topic     Partition: 0    Leader: 0       Replicas: 0     Isr: 0
        Topic: product-created-events-topic     Partition: 1    Leader: 0       Replicas: 0     Isr: 0
        Topic: product-created-events-topic     Partition: 2    Leader: 0       Replicas: 0     Isr: 0

I am using windows so I am using traditional kafka with zookeeper

Answer 1

using windows so I am using traditional kafka with zookeeper

You'll need to start 3 brokers on your machine to create any topics with 3 replicas. This means executing kafka-server-start multiple times, or using Docker Compose / minikube with 3 brokers