Search code examples
goconcurrencychannelgoroutine

Go routine:Making concurrent API requests


I am trying to understand channels and goroutines and tried to write a goroutine for making concurrent API requests to the server

But when I am running the code using a goroutine, it seems like it is taking the same time as it does without a goroutine.

func sendUser(user string, ch chan<- string)  {
    resp,err := http.get("URL"/user)
    //do the processing and get resp=string
    ch <- resp
}


func AsyncHTTP(users []string) ([]string, error) {
    ch := make(chan string)
    var responses []string
    var user string

    for _ , user = range users {
        go sendUser(user, ch)

        for {
            select {
            case r := <-ch:
                if r.err != nil {
                    fmt.Println(r.err)
                }
                responses = append(responses, r)
                **//Is there a better way to show that the processing of response is complete**?
                if len(responses) == len(users) { 
                    return responses, nil
                }
            case <-time.After(50 * time.Millisecond):
                fmt.Printf(".")
            }
        }
    }
    return responses, nil
}

Questions:

  1. Even though I am using a goroutine, request completion time is same as it is without goroutines? Is there anything I am doing wrong with goroutines?

  2. For telling the job not to wait anymore here I am using:

    if len(responses) == len(users)
    

    Is there a better way to show that the processing of response is complete and tell ch not to wait anymore?

  3. What is wait.Syncgroup? How can I use it in my goroutine?


Solution

  • I might do something like this..

    func sendUser(user string, ch chan<- string, wg *sync.WaitGroup) {
        defer wg.Done()
        resp, err := http.Get("URL/" + user)
        if err != nil {
            log.Println("err handle it")
        }
        defer resp.Body.Close()
        b, err := ioutil.ReadAll(resp.Body)
        if err != nil {
            log.Println("err handle it")
        }
        ch <- string(b)
    }
    
    func AsyncHTTP(users []string) ([]string, error) {
        ch := make(chan string)
        var responses []string
        var user string
        var wg sync.WaitGroup
        for _, user = range users {
            wg.Add(1)
            go sendUser(user, ch, &wg)
        }
    
        // close the channel in the background
        go func() {
            wg.Wait()
            close(ch)
        }()
        // read from channel as they come in until its closed
        for res := range ch {
            responses = append(responses, res)
        }
    
        return responses, nil
    }
    

    It allows to read from the channel as they are sent. By using a waitgroup I'll know when to close the channel. By putting the waitgroup and close in a goroutine I can read from the channel in "realtime" without blocking.