Say we were to use confluent-kafka-dotnet, ProducerConfig takes an array of Kafka broker addresses in its BootstrapServers property.
We could put those addresses in web.config, but assuming we're using a microservices architecture, if we were to change the Kafka cluster configuration (such as adding new brokers), we'd have to update a lot of web.config files.
I'm curious how others have solved this problem.
Rather than reading from a config file, I would suggest reading from an environment variable like BOOTSTRAP_SERVER, for example.
If you added a new broker to the existing cluster, then you don't have to worry about changing anything as long as your initial configs have at least 3 other healthy brokers listed - the rest will be bootstrapped by the Controller, and returned to client, hence the name of the property. You may need to bounce the app to get a new bootstrapping, or you can tweak the value of metadata.max.age.ms to re-force a refresh faster.
As mentioned in the comments, Consul is popular for dynamic service discovery, so you would then just have one entry for kafka.consul, and via DNS, it'll pick up any healthy service registered under kafka