Search code examples
pythonpython-multiprocessinggurobi

multiprocessing compatibility issue with Gurobi


The following simple multiprocessing of a square function works fine:

from multiprocessing import Pool

class A(object):

    def square(self, x):
        return x * x

    def test(self):
        pool = Pool()
        result = pool.map(self.square, range(5))
        return result

But when I add an initialization of a Gurobi model in the class like this,

from multiprocessing import Pool
from gurobipy import *

class A(object):

    def __init__(self):
        self.b = Model()

    def square(self, x):
        return x * x

    def test(self):
        pool = Pool()
        result = pool.map(self.square, range(5))
        return result

A().test()

It returns the following error:

File "model.pxi", line 290, in gurobipy.Model.__getattr__ (../../src/python/gurobipy.c:53411)
KeyError: '__getstate__'

A serial version with Gurobi works fine:

from gurobipy import *

class A(object):

    def __init__(self):
        self.b = Model()

    def square(self, x):
        return x * x

    def test(self):
        res = [self.square(i) for i in range(5)]
        return res

A().test()

Solution

  • The Gurobi API does not support sharing a single Gurobi environment across multiple processes or threads. If you want to solve multiple modules in multiple processes or threads, you must explicitly create multiple Gurobi environments in each process or thread.