Can anyone tell me, whether we can convert data frame to list in SparkR. I am aware that collect() function helps you do that. However, it is not advisable when we use large amount of data. In python/Scala, there is a function called local Iterator() which will convert the data frame to list. Am struggling with that in SparkR. Can anybody help!
Unfortunatelly collect() is the best method to do this. You can also try: saveAsTextFile but in that case you probably will not obtain whole data.