Python compute object property in separate task to improve performace

Question

I wonder if it's possible to compute an object property in a separate background thread when it's initialized to speed up my computation. I have this example code:

class Element:
   def __init__(self):
       self.__area = -1 # cache the area value
   
   @property
   def area(self)
       if self.__area < 0:
          self.__area = 2 # <- Here I put 2 as example but its slow area algorithm computation
       return self.__area

Now if I want to compute the total area of N elements.

total_area = sum(el.area for el in elements)

this works fine for a low number of elements but when the number of element increment I need a way to process this in parallel. I think it's the same for me to precompute the area rather than compute the total area in parallel.

Answer 1

The problem with Python and parallel computing is that there is that thing called GIL (Global Interpreter Lock). The GIL prevents a process to run multiple threads at the same time. So for that to work you would need to spawn a new process which has quiet some overhead. Furthermore it is cumbersome to exchange data between the processes.

There is that multiprocessing package which helps with creating new processes: https://docs.python.org/3/library/multiprocessing.html

Still in your case the question is, how do you split up the work. You could split up the elements in subblocks and create a process for each subblock which gets calculated there. The messaging between the processes can be implemented using pipes, as also described in 1. So you need to send a lot of objects to the new processes and then send them back via the pipes. I am not sure if the overhead of all that will make it faster in the end or even slower.