Search code examples
pythonmpihpcmpi4py

mpi4py getting only one MPI process available


I'm trying to use mpi4py but am getting the following error when trying to initialize it:

Tried to create an MPI pool, but there was only one MPI process available. Need at least two.

The value of MPI.COMM_WORLD.Get_size() is 1, which confirms the issue.

Still, when I run the usual test after installing it I get the expected output, which is weird:

$ mpiexec -n 5 python -m mpi4py.bench helloworld
Hello, World! I am process 0 of 5 on sevrhuntd1.
Hello, World! I am process 1 of 5 on sevrhuntd1.
Hello, World! I am process 2 of 5 on sevrhuntd1.
Hello, World! I am process 3 of 5 on sevrhuntd1.
Hello, World! I am process 4 of 5 on sevrhuntd1.

I'm not a system admin and it is taking them a while to process my request to remove openmpi and install mpich, as suggested here and here. Is there any other way around this error?

More specifically, I'm trying to create a pool using the MPIPool class in this file in the Platypus library and am getting the error here:

from mpi4py import MPI


class MPIPool(object):
    def __init__(self, comm=None, debug=False, loadbalance=False):
        self.comm = MPI.COMM_WORLD if comm is None else comm
        self.rank = self.comm.Get_rank()
        self.size = self.comm.Get_size() - 1
        self.debug = debug
        self.function = _error_function
        self.loadbalance = loadbalance
        if self.size == 0:
            raise ValueError("Tried to create an MPI pool, but there "
                             "was only one MPI process available. "
                             "Need at least two.") # This is the error.
    # More code below, but not making it there.

when trying to initialize it with the following in my main:

if __name__ == "__main__":
    pool = MPIPool()

Solution

  • This was a stupid mistake. I just had to call Python with mpiexec -n <# processes> python .... Problem solved.