MPI send and receive after task is completed. Is it even possible?

I am using the following code to run jobs in parallel. However, I was wondering if there was a way for a processor to send/receive a message after it completes a job. With this code, if I request 4 processors but have 56 job to complete, it only completes 4 jobs and then ends. I was wondering if there was a way to add to the code so all task are completed. I think a processor would need to send a message to the master processor telling it that the job is completed, send me another job if there are tjob left. I hope that makes sense. I am sorta stuck... Thanks in advance

Solution

sure you can... this is called a master/slave approach or dynamic parallelism. proc0 (the master, typically) will send out work to all the other procs. Then it will start waiting. As soon as one proc completes, it will collect the results and send back a new task, if available.

This kind of pseudocode:

if(myid.eq.0) then
  pending_tasks = 0
  sent_tasks = 0
  DO i=1,(numprocs-1)
    read(100,'(A)') command
    call MPI_SEND(command,200,MPI_CHAR,i,0,MPI_COMM_WORLD,ierr)
    pending_tasks = pending_tasks + 1
    sent_tasks = sent_tasks + 1
  ENDDO
  // all procs have one task to work on.
  DO
    // wait for results - from any source
    call MPI_RECV(result,200,MPI_CHAR,MPI_ANY_SOURCE,0,MPI_COMM_WORLD,istatus,ierr)
    free_proc = istatus(MPI_SOURCE)
    if (sent_tasks < nlines) then
      read(100,'(A)') command
      call MPI_SEND(command,200,MPI_CHAR,free_proc,0,MPI_COMM_WORLD,ierr)
      sent_tasks = sent_tasks + 1
    else
      // all tasks sent, but wait all the results
      pending_tasks = pending_tasks - 1
    endif
    call process_and_save_result(result)
    if (penging_tasks == 0) EXIT
  ENDDO
  // in this point the master can send out the 'QUIT' command to all the slaves
else
  DO
    call MPI_RECV(command,200,MPI_CHAR,0,0,MPI_COMM_WORLD,istatus,ierr)
    // that's a suggestion for a clean exit policy - pseudocode, don't forget
    if (command=='QUIT') EXIT
    call do_work(command, result)
    call MPI_SEND(result, 200,MPI_CHAR,0,0,MPI_COMM_WORLD,ierr)
  ENDDO
endif
call MPI_FINALIZE(ierr)

note: don't use the tag (set it to 0) if you don't need it.

see:

https://www.mcs.anl.gov/research/projects/mpi/mpi-standard/mpi-report-1.1/node35.htm

https://mpitutorial.com/tutorials/dynamic-receiving-with-mpi-probe-and-mpi-status/

Eventually proc0 can perform some task in the while, as an optimization, using mpi_irecv(), mpi_test(), mpi_wait().