How to maintain connection reuse during concurrent HTTP requests in Golang

Question

I want to make multiple HTTP requests in only one connection with using concurrency, but my code makes about four or five. But when I don't use concurrency, it will only have one connection.

package main

import (
    "fmt"
    "net/http"
    "net/url"
    "sync"
    "time"
)

func main() {
    proxyURL, err := url.Parse("http://localhost:9099")//In the traffic sniffer tool I see this program makes 5 requests
    if err != nil {
        fmt.Println(err)
        return
    }
    
    var wg sync.WaitGroup
    client := &http.Client{
        Transport: &http.Transport{
            MaxIdleConns:        10,
            IdleConnTimeout:     90 * time.Second,
            DisableKeepAlives:   false,
            Proxy: http.ProxyURL(proxyURL),
        },}

    for i := 0; i < 5; i++ {
        wg.Add(1)
        go func(index int) {
            defer wg.Done()
            resp, err := client.Get("https://www.httpbin.org/get")
            if err != nil {
                fmt.Println(err)
                return
            }
            err = resp.Body.Close()
            if err != nil {
                fmt.Println(err)
                return 
            }
            fmt.Println(index)
        }(i)
    }

    wg.Wait()
    fmt.Println("done")
}

Edit: Thanks to you, but I found another alternative solution: just establish a connecition before running goroutine, it's faster than setting "MaxConnsPerHost: 1" when there are lot of requests

Answer 1

You can consider using MaxConnsPerHost

// MaxConnsPerHost optionally limits the total number of
// connections per host, including connections in the dialing,
// active, and idle states. On limit violation, dials will block.
//
// Zero means no limit.
MaxConnsPerHost int

usage

Transport: &http.Transport{
   MaxConnsPerHost: 1,
   ..rest of config...
}

A note: if the remote server doesn't support keep-alive, you might not reuse the same connection but open a new one every time