I am trying to use a ProcessPoolExecutor, but I am getting the error "Queue objects should only be shared between processes through inheritance", but I am not using a Queue (at least not explicitly). I can't find anything that explains what I am doing wrong.
Here is some code that demonstrates the issue (not my actual code):
from concurrent.futures import ProcessPoolExecutor, as_completed
class WhyDoesntThisWork:
def __init__(self):
self.executor = ProcessPoolExecutor(4)
def execute_something(self, starting_letter):
futures = [self.executor.submit(self.something, starting_letter, d) for d in range(4)]
letter = None
for future in as_completed(futures):
letter = future.result()
print(letter)
def something(self, letter, d):
# do something pointless for the example
for x in range(d):
letter = chr(ord(letter) + 1)
if __name__ == '__main__':
WhyDoesntThisWork(). execute_something('A')
El Ruso has pointed out that making something() a staticmethod or classmethod makes the error go away. Unfortunately, my actual code needs to call other methods using self.
try this code for something
@staticmethod
def something(letter, d):
# do something pointless for the example
for x in range(d):
letter = chr(ord(letter) + 1)
or refactor to:
from concurrent.futures import ProcessPoolExecutor, as_completed
class WhyDoesntThisWork:
def something(self, letter, d):
# do something pointless for the example
for x in range(d):
letter = chr(ord(letter) + 1)
return letter
if __name__ == '__main__':
executor = ProcessPoolExecutor(4)
letter = 'A'
obj = WhyDoesntThisWork()
futures = [executor.submit(obj.something, letter, d) for d in range(4)]
for future in as_completed(futures):
print(future.result())