Search code examples
c++segmentation-faultopenmpi

MPI Seg fault in MPI_Comm_rank() using brew, git, or built in mpi

I cannot get MPI to work on my MacBook pro. In particular, it seg faults when I try to call MPI_Comm_rank(). Here is a sample program:

#include "mpi.h"
#include <iostream>

int main(int argc, char *argv[]) {
    std::cout << "Entered main\n";

    // Initialize parallel
    int rank, numProcess;
    MPI_Status status;
    MPI_Init(&argc, &argv);
    std::cout << "Init\n";
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    std::cout << "Rank\n";
    MPI_Comm_size(MPI_COMM_WORLD, &numProcess);

    std::cout << "Initialized MPI; rank " << rank << "\n";

    MPI_Finalize();
    return 0;
}

It compiles fine using mpic++ mpi_test.cpp -o mpi_test, but then I try to run it by calling mpirun -np 2 ./mpi_test and get the following error:

Entered main
Entered main
Init
Init
[Bens-MacBook-Pro:22004] *** Process received signal ***
[Bens-MacBook-Pro:22004] Signal: Segmentation fault: 11 (11)
[Bens-MacBook-Pro:22004] Signal code: Address not mapped (1)
[Bens-MacBook-Pro:22004] Failing at address: 0x10000004c
[Bens-MacBook-Pro:22004] [ 0] 0   libsystem_platform.dylib            0x00007fffc0118b3a _sigtramp + 26
[Bens-MacBook-Pro:22004] [ 1] 0   ???                                 0x0000000113f92978 0x0 + 4630063480
[Bens-MacBook-Pro:22004] [ 2] 0   mpi_test                            0x00000001078e40d1 main + 81
[Bens-MacBook-Pro:22004] [ 3] 0   libdyld.dylib                       0x00007fffbff09235 start + 1
[Bens-MacBook-Pro:22004] *** End of error message ***
[Bens-MacBook-Pro:22005] *** Process received signal ***
[Bens-MacBook-Pro:22005] Signal: Segmentation fault: 11 (11)
[Bens-MacBook-Pro:22005] Signal code: Address not mapped (1)
[Bens-MacBook-Pro:22005] Failing at address: 0x10000004c
[Bens-MacBook-Pro:22005] [ 0] 0   libsystem_platform.dylib            0x00007fffc0118b3a _sigtramp + 26
[Bens-MacBook-Pro:22005] [ 1] 0   ???                                 0x000000004fc26c50 0x0 + 1338141776
[Bens-MacBook-Pro:22005] [ 2] 0   mpi_test                            0x000000010ffd90d1 main + 81
[Bens-MacBook-Pro:22005] [ 3] 0   libdyld.dylib                       0x00007fffbff09235 start + 1
[Bens-MacBook-Pro:22005] [ 4] 0   ???                                 0x0000000000000001 0x0 + 1
[Bens-MacBook-Pro:22005] *** End of error message ***
-------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
-------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 1 with PID 0 on node Bens-MacBook-Pro exited on signal 11 (Segmentation fault: 11).
--------------------------------------------------------------------------

Notice it makes it past MPI_Init(), and fails in MPI_Comm().

I get the same error when I (i) download the most recent OpenMpi library from git and install it directly, and (ii) install OpenMPI via brew.

Solution

  • The problem seems to be due to conflicting versions of OpenMPI in the $PATH. When I reset my $PATH variable and specifically used the brew install for compilation and running, everything worked. At that point I uninstalled and removed all mpi libraries from my device, then reinstalled and linked the brew version. Now everything works. Posting in case somebody else runs in to this.