socket: too many open files Error for goroutines in indefinite loop

Question

I have a requirement in my program to send metrics to datadog indefinitely (for continuous app monitoring in datadog). The program runs for a while and exits with the error "dial udp 127.0.0.1:18125: socket: too many open files".

    func sendData(name []string, channel chan []string) {
      c, err := statsd.New("127.0.0.1:18125")
      if err != nil {
        log.Fatal(err)
      }

      v := versionDetails()
      tag := "tag:" + v
      final_tag := []string{dd_tags}
      appEpochTimeList := epochTime()
      rate := float64(1)

      for i, app := range name {
        e := c.Gauge(app, float64(appEpochTimeList[i]), final_tag , rate)
        if e != nil {
            log.Println(e)
            channel <- name
        }
        channel <- name
        log.Printf("Metrics Sent !!")
      }
  }

The app names are read from a config.toml file

Answer 1

The problem is your sendData() function. This function is called in your for loop and has the following line:

c, err := statsd.New("127.0.0.1:18125")

This line will create a new DataDog client, which uses a Unix socket. This explains your error message.

With every iteration of your loop, a new socket is "allocated". After a sufficient amount of loops no sockets can be opened, resulting in:

socket: too many open files

To fix this you should create the client only once and pass it to your method as parameter.

func sendData(client *statsd.Client, name []string, channel chan []string) {
    // do something with client...
}

func main() {
    client, err := statsd.New("127.0.0.1:18125")
    if err != nil {
        log.Fatal(err)
    }

    // do something else ...

    for res := range channel {
        go func(client *statsd.Client, appName []string) {
            time.Sleep(5 * time.Second)
            go sendData(client, appName, channel)
        }(client, res)
    }
}