Search code examples
apache-sparkapache-spark-sqlspark-streaming

How to get non-null sorted ascending data from Spark DataFrame?


I load the data into data frames where one of the columns is zipCode (String type). I wonder how to get non-null values for that column in ascending order in Scala? Many thanks in advance.


Solution

  • scala> val df = Seq("2", "1", null).toDF("x")
    df: org.apache.spark.sql.DataFrame = [x: string]
    
    scala> df.orderBy($"x".asc_nulls_last).show
    +----+
    |   x|
    +----+
    |   1|
    |   2|
    |null|
    +----+