How can we run connect-standalone with the same connector with two different configurations?

Question

I am using a sink connector with two different sinks(of the same type), the connector needs to be invoked with two different configuration. Currently, we are dealing with that using screen:

launchScript.sh

screen -dmS conn-one runConnector.sh config/connect-sink-conn-one.properties config/conn-one.properties
screen -dmS conn-two ./bin/runConnector.sh config/conn-two.properties config/conn-two.properties

runConnector.sh

connect-standalone $1 $2

Is there a way of achieving this without having to use GNU Screen?

Answer 1

The bin/connect-standalone.sh script can take multiple properties files with different connector configurations. You just have to use different names for each connector. So for example you can do something like this:

bin/connect-standalone.sh config/connect-standalone.properties config/connect-file-sink.properties config/connect-file-sink2.properties

Where config/connect-file-sink.properties is:

name=local-file-sink
connector.class=FileStreamSink
tasks.max=1
file=test.sink.txt
topics=connect-test

and config/connect-file-sink2.properties is:

name=local-file-sink2
connector.class=FileStreamSink
tasks.max=1
file=test.sink.txt2
topics=connect-test