In Apache Spark API I can use startsWith
function in order to test the value of the column:
myDataFrame.filter(col("columnName").startsWith("PREFIX"))
Is it possible to do the same in Spark SQL expression and if so, could you please show an example?.
I found the following solution that works with Spark SQL query without custom UDP and out of the box, for example:
CASE WHEN instr(PRICE_SOURCE_INDICATOR,'MAC') = 1