Search code examples
spring-cloud-dataflow

Overriding properties in Spring Cloud Dataflow stream definition


For Kafka streams (consumers) that use KafkaAvroDeserializer I need to override some configs on deployment with --propertiesFile to work. KafkaBinder 1.1.1 added this config option.

I prefer not to use deployment properties if I can, because they're not visible on the dashboard and it's dangerous if another user decides to stop and restart a stream without setting the properties. Is it possible to include these properties within the stream definition? My repeated attempts have failed to work. I don't see an example of this on the reference doc.

e.g. for yaml equivalent:

<...>
binder:
  configuration:
    value.deserializer: io.confluent.kafka.serializers.KafkaAvroDeserializer
    schema.registry.url: http://my.dns.com:8081

Solution

  • You can specify those Spring Cloud Stream properties in Data Flow Stream definition DSL like this:

    stream create a1 --definition "time --spring.cloud.stream.kafka.binder.configuration.value.serializer=serializer | log --spring.cloud.stream.kafka.binder.configuration.value.deserializer=deserializer --spring.cloud.stream.kafka.binder.configuration.schema.registry.url='http://myregistry'" --deploy