I am currently working on a chess engine, and I am looking at parallelization as a way to optimize the time it takes to search legal moves to find the best one. I found the Python module multiprocessing, however since it needs to execute in "name"=="main" how would I use it in my current class model? My code looks something like this:
class ourEngine:
def __init__(self, board, engine_color, thinking_time=0):
self.board = board
self.thinking_time = thinking_time
self.engine_color = engine_color
def evaluate(self, board):
# Evaluation calculation
def search(self, board, depth, engine_color, alpha=float("-inf"), beta=float("inf")):
# Minimax algorithm with alpha-beta pruning for every move in python-chess board.legal_moves
How would I implement the Pool from the multiprocessing library to process legal_moves simultaneously on all cpu's without having an init? I also realize I will have to set up a queue, but I can't find an example online anywhere
I tried implementing the module in a separate function which would then call search but it did not work.
Edit: Fixed typos
On platforms that use the spawn method of creating child processes (for example, Windows), to launch a child process Python launches a new Python interpreter into the new process's address space that re-processes the source code executing everything at global scope (import statement, global variables, function definitions, class definitions, etc.) to initialize memory and only then executes the function/method you specified to initially execute in the new process.
The problem with this scheme is that any statement at global scope that directly creates a child process or indirectly does so (e.g. by calling a function that ultimately creates a child process) would result in a recursive child process-creating loop (this doesn't actually happen since Python detects this attempt to create a new child process during this memory-initialization stage).
To get around this problem, any such direct or indirect process-creating statement must be placed in a if __name__ == '__main__': block. The recursion will be prevented since __name__ will not be '__main__' in any child process and therefore the process-creating statement will not be executed.
So imagine you modified your search method so that it creates child processes among which it distributes the searching. So any code at global scope that ultimately ends up calling this method needs to be in that special if block. For example:
class OurEngine:
... # Code omitted for brevity
def create_engine_and_search():
... # Code omitted
engine = OurEngine(board, some_color, some_thinking_time_
...
engine.search()
def main():
...
create_engine_and_search()
# Invoice main:
main()
The call to main is at global scope. As a result of calling main the call to OurEngine.search will ultimately be made and we postulated that the search method creates child processes. To initialize memory for these new processes, this file will be re-interpreted in the new process and main, which is at global scope will be called again. This is a problem! But all we need to do is:
...
# Only call main if we are the main process:
if __name__ == '__main__':
main()
What is your platform? We must tag all multiprocessing questions with our current platform since answers depend on the platform. If you were running under Linux, for example, a different mechanism is used to initialize memory for the new child processes and we do not need a special if test.