Search code examples
c++boostmpi

Operation type in boost mpi collective operations


In boost::mpi some collective operations, such as reduce, require an operation to be passed to the routine. I am not sure what exactly should be the type of this operation.

The following minimal example compiles without warnings.

#include <iostream>
#include <boost/mpi/collectives.hpp>

int mysum(int a, int b) { return a + b; }

int main()
{
  boost::mpi::environment env;
  boost::mpi::communicator world;

  int data = world.rank() + 1;
  int data_reduced;
  boost::mpi::reduce(world, data, data_reduced, mysum, 0);
  if (world.rank() == 0)
    std::cout << data_reduced << std::endl;

  return 0;
}

However, running with more than 1 task crashes

$ mpirun -n 2 ./mpi
munmap_chunk(): invalid pointer
...

Altering the code as follows make the program work without crashes.

#include <iostream>
#include <boost/mpi/collectives.hpp>

struct mysum {
  int operator()(int a, int b) { return a + b; }
};

int main()
{
  boost::mpi::environment env;
  boost::mpi::communicator world;

  int data = world.rank() + 1;
  int data_reduced;
  boost::mpi::reduce(world, data, data_reduced, mysum{}, 0);
  if (world.rank() == 0)
    std::cout << data_reduced << std::endl;

  return 0;
}

(I am aware that this is equivalent to std::plus, the program is just an example)

$ mpirun -n 2 ./mpi
3

What is the difference, and why is the second version working?

Edit

The question also arises because both variants of mysum can be called as mysum(....), i.e., both are callable. So in both cases a code like the following works.

template <class Callable, class ArgType>
auto dosomething(ArgType a, ArgType b, Callable&& L) { return L(a, b); }

auto x = dosomething(mysum, 1, 2);

(This is essentially equivalent to std::invoke)


Solution

  • It looks like this is what boost::mpi expects!

    If you mimic boost/mpi/operations.h, your user defined reduction operator can be written like this

    template<typename T>
    struct mysum
    {
      typedef T first_argument_type;
      typedef T second_argument_type;
      typedef T result_type;
      const T& operator()(const T& x, const T& y) const
      {
        return x + y;
      }
    };
    

    and then your reduce invocation will look like that

    boost::mpi::reduce(world, data, data_reduced, std::plus<int>(), 0);
    

    bottom line, even if your mysum is the right thing to pass to the plain C MPI_Op_create(), this is not what boost::mpi expects.