Search code examples
pythonapache-sparkpysparkrdd

unable to write data to RDD


I am running pyspark on windows and implementing an algorithm that was previously implemented in python

I have converted the list to RDD but got the error

voxel =[[ ['0' for col in range(100)] for col in range(100)] for row in range(6)] 
delay = sc.parallelize(voxel,4)
locationforant=[[75,15],[100,50],[75,80],[25,80],[0,50],[25,15]]
for a in range(6):
    for x in range(100):
        for y in range(100):
            distance=((math.sqrt((x-locationforant[a][0])**2+(y-locationforant[a][1])**2))/100)*200
            delay[a][x][y] = round(distance)

TypeError                                 Traceback (most recent call last)
<ipython-input-9-b81a67e36764> in <module>()
     21             distance=((math.sqrt((x-locationforant[a][0])**2+(y-locationforant[a][1])**2))/100)*200
     22 #             breakpoint()
---> 23             delay[a][x][y] = round(distance)

TypeError: 'RDD' object does not support indexing

Solution

  • As the error indicates, you cannot use the index operator ([]) on an RDD. You'll need to use something like map to apply a function to each row.