I have a python dict, which is a config to a job which looks like this:
#Values
varX = "x"
varY = "y"
conf={
"spark.someVal": '{{ someComplexMethod("test") }}'
}
Problem is that when I want to replace/modify the returned value from the method with some globally declared values, it does not reflect since it is probably not in scope.
for eg, This works:
conf={
"spark.someVal": '{{ someComplexMethod("test").replace("x","y") }}'
}
But below lines don't.
conf={
"spark.someVal": '{{ someComplexMethod("test").replace(varX, varY) }}'
}
OR
conf={
"spark.someVal": '{{ someComplexMethod("test") }}'.replace(varX, varY)
}
I don't want to use values directly, any way around this? And can someone please explain what is the use of curly braces here.
Maybe you can setup varX
and varY
as Airflow Variables.
Then you can use them in your Jinja templates like so:
'{{ someComplexMethod("test").replace(var.value.varX, var.value.varY) }}'