When attempting to execute the following spark code in Databricks I get the error:
NameError: name 'expr' is not defined
%python
df = sql("select * from xxxxxxx.xxxxxxx")
transfromWithCol = (df.withColumn("MyTestName", expr("case when first_name = 'Peter' then 1 else 0 end")))
Any thoughts
Looks like you forgot to import "expr" first from the library. Following is the import statement to avert the encountered error:
from pyspark.sql.functions import expr