Spark-submit Spring Boot profiles.active variable

Question

Good day for you collegues. I am trying run my appliacation on a cluster. My script looks like:

#!/usr/bin/env bash
/usr/hdp/current/spark2-client/bin/spark-submit \
--driver-memory 12G \
--master yarn \
--driver-java-options -Dspring.profiles.active=vf \
--driver-java-options -Doperation.localDate=2018-06-01 \
/mnt/store/vm/base-operations/my.jar

But when my app starts, Spring Boot shows to me next:

18/07/10 16:34:19 INFO OperationRunner: The following profiles are active: hadoop

Then i try to start it like:

 java -jar -Dspring.profiles.active=vf multirating-bigdata-operations-MASTER-SNAPSHOT.jar

I see:

18/07/10 16:34:19 INFO OperationRunner: The following profiles are active: vf

Whats wrong with spark-submit? How to force needed applicataion-{suffix}.yml for start app?

Answer 1

It works with double quotes in one line.

#!/usr/bin/env bash
/usr/hdp/current/spark2-client/bin/spark-submit \
--driver-java-options "-Dspring.profiles.active=vf -Doperation.localDate=2018-06-01" \
--driver-memory 12G \
--master yarn \
/mnt/store/viewing-maker/base-operations/multirating-bigdata-operations-MASTER-SNAPSHOT.jar