I am using flink-docker 1.4.0 and I have created a scala job to be run on Flink. The job uses typesafe config which looks similar to that :
servers = "localhost:9092"
servers = ${?KAFKA_SERVERS}
The problem is that it works perfectly, if I set the KAFKA_SERVERS in the docker-compose file starting the jobmanager and taskmanager.
But I would like to set them from WebUI using Program Arguments.
I have tried to add env.java.opts="-DKAFKA_SERVERS=192.168.99.100:9092 like in
this image.But it is not visible inside jar. Is there a way to set environment variables per job in Flink WebUI ?
It will not work since TaskManager's JVM machines are spawned before submitting your job and job is executed inside those JVMs, thus you can not pass environment variables in that way. The box where you put env.java.opts="-DKAFKA_SERVERS=192.168.99.100:9092" is even called "Program options" (args passed to your main function).
This answer provides more detailed explanation to this: Apache Flink - custom java options are not recognized inside job and suggests solution:
Do not relay on environment variables when spawning your job.
