Search code examples
pythonapache-sparkpysparkdistributed-computingrdd

Convert a simple one line string to RDD in Spark


I have a simple line:

line = "Hello, world"

I would like to convert it to an RDD with only one element. I have tried

sc.parallelize(line)

But it get:

sc.parallelize(line).collect()
['H', 'e', 'l', 'l', 'o', ',', ' ', 'w', 'o', 'r', 'l', 'd']

Any ideas?


Solution

  • try using List as parameter:

    sc.parallelize(List(line)).collect()
    

    it returns

    res1: Array[String] = Array(hello,world)