Search code examples
goconcurrencygoroutine

How to properly configure the gorutine limit?


I have a task where three functions are performed if they are selected to work. I want to limit the number of horoutines per function. For example so that each of them has only a maximum of 10 goroutines running.

func main() {
    checkMethod1 := true
    checkMethod2 := false
    checkMethod3 := true

    list := []string{"info1", "info2", "info3", "info5"}

    for _, value := range list {
        value := value
        if checkMethod1 {
            go func() {
                //use value
                fmt.Println(value)
            }()
        }
        if checkMethod2 {
            go func() {
                //use value
                fmt.Println(value)
            }()
        }
        if checkMethod3 {
            go func() {
                //use value
                fmt.Println(value)
            }()
        }
    }
    //finish
    fmt.Println("All done")
}

I know that you can limit the number of goroutines to a pool of workers. But if I make one pool of workers with a limit of 10 goroutines, that number is divided by 3 tasks, and I need each function to have 10 goroutines.

I could create 3 pools, but that doesn't seem like a workable way to me.

I would like to use this library to create working pools: https://github.com/sourcegraph/conc


Solution

  • Here's a way to do it: use a buffered channel for each option, so you can limit active goroutines:

    m1:=make(chan struct{},10)
    m2:=make(chan struct{},10)
    m3:=make(chan struct{},10)
    wg:=sync.WaitGroup{}
    
    for _, value := range list {
       value := value
       if checkMethod1 {
           m1<-struct{}{}
           wg.Add(1)
           go func() {
               defer func() { 
                  <-m1 
                  wg.Done()
               }()
               // do work
           }()
       }
       if checkMethod2 {
           m2<-struct{}{}
           wg.Add(1)
           go func() {
               defer func() { 
                   <-m2 
                   wg.Done()
               }()
               // do work
           }()
       }
       ...
    
      wg.Wait()
    }