my data set looks like this
I wrote this code:
flightData2015.select("*",when(flightData2015['count']>200,'above200')
.when(flightData2015['count']>400,'above400').otherwise("below").alias("new count")).show()
output : red line does not follow my logic, I want to know why the second "when" condition is not working
Firstly, the condition ">200" will satisfy items that are greater than 400 also, so that is why the second when is not used.
Secondly, a nested when otherwise clause should work.
when(flightData2015['count']>400,'above400').otherwise(when(flightData2015['count']>200,'above200').otherwise("below"))