KAFKA clients (producer & consumer) discover the topology by connecting to one the configured bootstrap server and from then on it will work with those set of broker nodes in the cluster directly.
With that standard thing, can we change producer and consumer without restarting them to work with different cluster by changing the bootstrap servers? Can we change the bootstrap server Config without restarting the clients?
There is a client config called metadata.max.age.ms which will force the client to refresh the broker topology but that covers only new partition leader and new broker node added to the cluster.
This is mainly to cover the scenario where KAFKA Cluster in a DC is down for some unexpected reason and clients from the same DC who already established connection with the same DC KAFKA cluster and want to explore the option of redirecting already started KAFKA clients to work with other DC KAFKA cluster in case same DC cluster is down.
You cannot change the configured bootstrap servers in the official Kafka Client.
Clients are able to discover the full Kafka cluster from the initial bootstrap servers and will periodically refresh this data (every metadata.max.age.ms
) but they cannot be moved to another cluster on the fly.
As you mentioned, clients will need to be restarted pointing to the new cluster.