Search code examples
fortranopenmp

Is omp barrier equivalent to omp end parallel in Fortran

My question is about synchronizing threads. Basically, if I have an OpenMP code in Fortran, each thread is doing something. There are two possibilities for synchronizing them (let some variable have the same value in each thread), I think.

  1. add !$OMP BARRIER
  2. add !$OMP END PARALLEL. If necessary, add !$OMP PARALLEL and !$OMP END PARALLEL block later on.

Are options 1) and 2) equivalent? I saw some question about barrier in nested threads omp barrier nested threads So far I am more interseted in simpler scanarios with Fortran. E.g., for the code below, if I use barrier, it seems the two if (sum > 500) then conditions will behave the same, at least by gfortran.

PROGRAM test
USE OMP_LIB

integer :: numthreads, i, sum



numthreads = 2
sum = 0

call omp_set_num_threads(numthreads)

!$OMP PARALLEL


  if (OMP_GET_THREAD_NUM() == 0) then
    write (*,*) 'a'
    
    do i = 1, 30
      write (*,*) sum
      sum = sum +  i
    end do 

    !write (*,*) 'sum', sum
  else if  (OMP_GET_THREAD_NUM() == 1) then
    write (*,*) 'b'
    do i = 1, 15
      write (*,*) sum
      sum = sum +  i
    end do  
    
    !write (*,*) 'sum', sum    
  end if  

!$OMP BARRIER

  if (sum > 500) then
    write (*,*) 'sum v1'
  else
    write (*,*) 'not yet v1'
  end if

!$OMP END PARALLEL

 
  if (sum > 500) then
    write (*,*) 'sum v2', sum
  else
    write (*,*) 'not yet v2', sum
  end if
 

END

My concern is, for a code

blah1 
!$OMP PARALLEL 
!$OMP END PARALLEL
blah2

if the computer will execute as blah1 -> omp -> blah2. If the variables (e.g., the sum in the example code) in blah2 has been evaluated completely in the omp block, I don't need to worry if some thread in omp goes faster, compute part of an entry (e.g., sum in the question), and goes to the if condition in blah2 section, leads to some unexpected result.

Solution

  • No, they are not equivalent at all.

    For !$omp end parallel let's think a little bit about how parallelism works within OpenMP. At the start of your program you just have a single so called master thread available. This remains the case until you reach a parallel region, within which you have multiple threads available, the master and (possibly) a number of others. In Fortran a parallel region is started with the !$omp parallel directive. It is closed by a !$omp end parallel directive, after which you just have the master thread available to your code until you start another parallel region. Thus !$omp end parallel simply marks the end of a parallel region.

    Within a parallel region a number of OpenMP directives start to have an affect. One of these is !$omp barrier which requires that a given thread waits at that point in the code until all threads have reached that point (for a carefully chosen value of "all" when things like nested parallelism is in use - see the standard at https://www.openmp.org/spec-html/5.0/openmpsu90.html for more details). !$omp barrier has nothing to do with delimiting parallel regions. Thus after its use all threads are still available for use, and outside of a parallel region it will have no effect.

    The following little code might help illustrate things

    ijb@ijb-Latitude-5410:~/work/stack$ cat omp_bar.f90
Program omp_bar

  !$ Use omp_lib, Only : omp_get_num_threads, omp_in_parallel

  Implicit None

  Integer n_th
  
  !$omp parallel default( none ) private( n_th )
  n_th = 1
  !$ n_th = omp_get_num_threads()
  Write( *, * ) 'Hello at 1 on ', n_th, ' threads. ', &
       'Are we in a parallel region ?', omp_in_parallel()
  !$omp barrier
  Write( *, * ) 'Hello at 2', omp_in_parallel()
  !$omp end parallel

  Write( *, * ) 'Hello at 3', omp_in_parallel()

End Program omp_bar
ijb@ijb-Latitude-5410:~/work/stack$ gfortran --version
GNU Fortran (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

ijb@ijb-Latitude-5410:~/work/stack$ gfortran -fopenmp -std=f2008 -Wall -Wextra -fcheck=all -O -g omp_bar.f90 
ijb@ijb-Latitude-5410:~/work/stack$ ./a.out
 Hello at 1 on            2  threads. Are we in a parallel region ? T
 Hello at 1 on            2  threads. Are we in a parallel region ? T
 Hello at 2 T
 Hello at 2 T
 Hello at 3 F

    [Yes, I know the barrier is not guaranteed to synchronise the output order, I got lucky here]