Search code examples
pysparkcasemultiple-conditions

Multiple when conditions is not working Pyspark


my data set looks like this

raw data

I wrote this code:

flightData2015.select("*",when(flightData2015['count']>200,'above200')
                  .when(flightData2015['count']>400,'above400').otherwise("below").alias("new count")).show()

output : red line does not follow my logic, I want to know why the second "when" condition is not working


Solution

  • Firstly, the condition ">200" will satisfy items that are greater than 400 also, so that is why the second when is not used.

    Secondly, a nested when otherwise clause should work.

    when(flightData2015['count']>400,'above400').otherwise(when(flightData2015['count']>200,'above200').otherwise("below"))