Search code examples
apache-kafkahdp

kafka + how to add additional partitions to the existing XX partitions?

this is example how to create new 10 topic partitions with name - test_test

kafka-topics.sh --create --zookeeper zookeeper01:2181 --replication-factor 3 --partitions 10 --topic test_test

Created topic "test_test".


[root@kafka01 kafka-data]# \ls -ltr | grep  test_test
drwxr-xr-x 2 kafka hadoop  4096 Mar 22 16:53 test_test-8
drwxr-xr-x 2 kafka hadoop  4096 Mar 22 16:53 test_test-5
drwxr-xr-x 2 kafka hadoop  4096 Mar 22 16:53 test_test-2
drwxr-xr-x 2 kafka hadoop  4096 Mar 22 16:53 test_test-0
drwxr-xr-x 2 kafka hadoop  4096 Mar 22 16:53 test_test-7
drwxr-xr-x 2 kafka hadoop  4096 Mar 22 16:53 test_test-4
drwxr-xr-x 2 kafka hadoop  4096 Mar 22 16:53 test_test-1
drwxr-xr-x 2 kafka hadoop  4096 Mar 22 16:53 test_test-9
drwxr-xr-x 2 kafka hadoop  4096 Mar 22 16:53 test_test-6
drwxr-xr-x 2 kafka hadoop  4096 Mar 22 16:53 test_test-3

now we want to add additional 10 partitions to the topic name - test_test

how to add additional partitions to the existing 10 partitions ?

Solution

  • You can run this command: 

    ./bin/kafka-topics.sh --alter --bootstrap-server localhost:9092 --topic test_test --partitions 20

    By the way there are two things to consider about changing partitions:

    • Decreasing the number of partitions is not allowed
    • If you add more partitions to a topic, key based ordering of the messages cannot be guaranteed

    Note: If your Kafka version is older than 2.2 you must use --zookeeper parameter instead of --bootstrap-server