Search code examples
dockerdocker-composetelegrafulimit

How to increase memory lock in Telegraf container?


I run Telegraf as a Docker container with a docker-compose file.

telegraf:
    image: telegraf:1.26.3
    container_name: telegraf
    depends_on:
      socket-proxy:
        condition: service_started
      influxdb:
        condition: service_healthy
    hostname: telegraf
    volumes:
      # several volumes
    ports:
      # several ports
    networks:
      - socket
      - data_export
    restart: unless-stopped
    labels:
      <<: *default-labels
      diun.include_tags: "^latest$$;^1.26.3$$"

It's working fine but I've added a few influxdb connections ([[outputs.influxdb]]) and when the container starts a warning shows off:

2023-06-10T14:25:16Z I! Found 14 secrets...
2023-06-10T14:25:16Z W! Insufficient lockable memory 64kb when 112kb is required. Please increase the limit for Telegraf in your Operating System!

I tried to exec ulimit in the container but it doesn't work.

root@telegraf:/# ulimit -l    
64
root@telegraf:/# ulimit -l 112
bash: ulimit: max locked memory: cannot modify limit: Operation not permitted

I tried to add ulimits in my docker-compose file but I get a panic error

ulimits:
  memlock: 112
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xa0525f]
goroutine 1 [running]:
github.com/awnumar/memguard/core.Purge.func1(0xc0001df930)
    /go/pkg/mod/github.com/awnumar/[email protected]/core/exit.go:23 +0x3f
github.com/awnumar/memguard/core.Purge()
    /go/pkg/mod/github.com/awnumar/[email protected]/core/exit.go:51 +0x25
github.com/awnumar/memguard/core.Panic({0x5be2400, 0xc0002a2330})
    /go/pkg/mod/github.com/awnumar/[email protected]/core/exit.go:85 +0x25
github.com/awnumar/memguard/core.NewBuffer(0x20)
    /go/pkg/mod/github.com/awnumar/[email protected]/core/buffer.go:73 +0x2d5
github.com/awnumar/memguard/core.NewCoffer()
    /go/pkg/mod/github.com/awnumar/[email protected]/core/coffer.go:30 +0x34
github.com/awnumar/memguard/core.init.0()
    /go/pkg/mod/github.com/awnumar/[email protected]/core/enclave.go:15 +0x2e

How can I increase the max locked memory for this container?


Solution

  • I don't know this for certain, but I think the memlock value is interpreted as bytes, so to specify a limit of 112KB you need to write:

    services:
      telegraf:
        ulimits:
          memlock: 114688
    

    A test seems to confirm my interpretation; given this complete compose file:

    services:
      telegraf:
        ulimits:
          memlock: 114688
        image: telegraf:1.26.3
        entrypoint:
          - sleep
          - inf
    

    After I docker-compose up the stack, I can exec into the telegraf container and run ulimit -a and see that the "locked memory" ulimit shows up as 112.

    $ docker-compose exec telegraf sh -c 'ulimit -a'
    time(seconds)        unlimited
    file(blocks)         unlimited
    data(kbytes)         unlimited
    stack(kbytes)        8192
    coredump(blocks)     unlimited
    memory(kbytes)       unlimited
    locked memory(kbytes) 112
    process              unlimited
    nofiles              1073741816
    vmemory(kbytes)      unlimited
    locks                unlimited
    rtprio               0