How to flatten RDD which contains sub list into main list

val rdd = df.rdd.map(
      line => Row(
        "BNK",
        format.format(Calendar.getInstance().getTime()),
        line(0),
        scala.xml.XML.loadString("<?xml version='1.0' encoding='utf-8'?>" + line(1)).child.map(_.text).filter(_.nonEmpty)


      )
    )

Produce the output

 values = {Object[4]@9906} 
 0 = "BNK"
 1 = "18-3-2019"
 2 = "185687194277431.060001"
 3 = {$colon$colon@9910} "::" size = 20
  0 = "KH0010001"
  1 = "-1171035537.00"
  2 = "9"
  3 = "65232"
  4 = "1"
  5 = "KHR"
  6 = "TR"
  7 = "6-54-10-1-005-004"
  8 = "1"
  9 = "1"
  10 = "DC183050001002108"
  11 = "DC"
  12 = "20181101"
  13 = "185687194277431.06"
  14 = "1"
  15 = "1"
  16 = "5022_DMUSER__OFS_DM.OFS.SRC.VAL"
  17 = "1811012130"
  18 = "6012_DMUSER"
  19 = "PL.65232.......1.....KH0010001"

How can flatten the values[3] is sub list with 20 items in to main list.

so the expected output:

 values = 
 0 = "BNK"
 1 = "18-3-2019"
 2 = "185687194277431.060001"
 3 = "KH0010001"
 4 = "-1171035537.00"
 5 = "9"
 6 = "65232"
 7 = "1"
 ..

Solution

Another try after updated question. I think the schema needs to be manually generated as the values are based on a list. Assuming the list is always of size 20:

val schema = StructType((0 to 22)
  .map(x => StructField(x.toString, IntegerType))
  .toList)
spark.createDataFrame(df.rdd.map(line => Row.fromSeq("BNK" :: format.format(Calendar.getInstance().getTime()) :: line(0) :: scala.xml.XML.loadString("<?xml version='1.0' encoding='utf-8'?>" + line(1)).child.map(_.text).filter(_.nonEmpty).toList)), schema)

The list needs to be capped/padded if it is not always of size 20. Hope it helps.