Search code examples
elasticsearchnginxfluentdefk

Nginx cannot connect fluentd in EFK stack


I am setting up a stack with an application consisting of nginx, redis, mysql, myapp. Nginx proxies requests to myapp. I want to send logs from nginx to EFK stack, but an error occurs when starting the nginx service:

Error response from daemon: dial tcp 127.0.0.1:24224: connect: connection refused

docker-compose.yml for stack with myapp

version: "3.8"
services:
  
  nginx:
    image: nginx:alpine
    deploy:
      mode: replicated
      replicas: 2
      labels:
        - traefik.enable=true
        - traefik.http.routers.node1.rule=Host(`${NODE1}`)
        - traefik.http.routers.node1.service=nginx
        - traefik.http.routers.node2.rule=Host(`${NODE2}`)
        - traefik.http.routers.node2.service=nginx
        - traefik.http.routers.node3.rule=Host(`${NODE3}`)
        - traefik.http.routers.node3.service=nginx
        - traefik.http.services.nginx.loadbalancer.server.port=80
      placement:
        constraints:
          - node.role == manager
    logging:
      driver: fluentd
      options:
        fluentd-address: localhost:24224
        tag: nginx-
    volumes:
      - ./nginx/default.conf:/etc/nginx/conf.d/default.conf
    ports: 
      - 80:80
    depends_on: 
      - myapp
    networks:
      - traefik-public
...

All stacks are in the same traefik-public network, if you make ping fluentd from any container, fluentd responds

Part of efk.yml

version: "3.7"
services:

  fluentd:
    image: registry.rebrainme.com/docker_users_repos/3912/dkr-30-voting/fluentd
    deploy:
      mode: global
    volumes:
      - /mnt/fluent.conf:/fluentd/etc/fluent.conf
    ports:
      - "24224:24224"
      - "24224:24224/udp"
    depends_on:
      - elasticsearch
      - kibana
    networks:
      - traefik-public
...

fluent.conf

<source>
  @type forward
  port 24224
  bind localhost
</source>

<match *.**>
  @type copy

  <store>
    @type elasticsearch
    host elasticsearch
    port 9200
    logstash_format true
    logstash_prefix fluentd
    logstash_dateformat %Y%m%d
    include_tag_key true
    type_name access_log
    tag_key @log_name
    flush_interval 1s
  </store>

  <store>
    @type stdout
  </store>
</match>

I ask for help


Solution

  • In order for there to be a connection between the efk stack and the application stack, it is necessary to bind the fluentd port to the host port

    version: "3.7"
    services:
    
      fluentd:
        image: my_fluentd_image:latest
        deploy:
          mode: global
        configs:
          - source: fluent-conf
            target: /fluentd/etc/fluent.conf
        ports:
          - target: 24224
            published: 24224
            protocol: tcp
            mode: host
        depends_on:
          - elasticsearch
          - kibana
        networks:
          - traefik-public