How to connect metricbeat to elasticsearch and kibana with docker

I've setup elasticsearch and kibana with docker compose. elasticsearch is deployed on: localhost:9200 while kibana is deployed on localhost:5601

When trying to deploy metricbeat with docker run I got the following errors:

$ docker run docker.elastic.co/beats/metricbeat:6.3.2 setup -E setup.kibana.host=kibana:5601 -E output.elasticsearch.hosts=["localhost:9200"]

Exiting: Couldn't connect to any of the configured Elasticsearch hosts. Errors: [Error connection to Elasticsearch http://localhost:9200: Get http://localhost:9200: dial tcp [::1]:9200: connect: cannot assign requested address]

Exiting: Couldn't connect to any of the configured Elasticsearch hosts. Errors: [Error connection to Elasticsearch http://elasticsearch:9200: Get http://elasticsearch:9200: lookup elasticsearch on 192.168.65.1:53: no such host]

My docker-compose.yml:

# ./docker-compose.yml

version: "3.7"
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.3.2
    environment:
#      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - elkdata:/usr/share/elasticsearch/data
    ports:
      - "9200:9200"
    restart: always
  kibana:
    image: docker.elastic.co/kibana/kibana:6.3.2
    volumes:
      - kibana:/usr/share/kibana/config
    ports:
      - "5601:5601"
    depends_on:
      - elasticsearch
    restart: always
volumes:
  elkdata:
  kibana:

Solution

First edit your docker-compose file by adding a name for default docker network:

version: "3.7"
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.3.2
    environment:
#      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - elkdata:/usr/share/elasticsearch/data
    ports:
      - "9200:9200"
    networks:
      - my-network
    restart: always
  kibana:
    image: docker.elastic.co/kibana/kibana:6.3.2
    volumes:
      - kibana:/usr/share/kibana/config
    ports:
      - "5601:5601"
    networks:
      - my-network
    depends_on:
      - elasticsearch
    restart: always
volumes:
  elkdata:
  kibana:
networks:
  my-network:
    name: awesome-name

Execute docker-compose up and then start metricbeat with the below command:

$ docker run docker.elastic.co/beats/metricbeat:6.3.2 --network=awesome-name setup -E setup.kibana.host=kibana:5601 -E output.elasticsearch.hosts=["elasticsearch:9200"]

Explanation:

When you try to deploy metricbeat, you provide below envars:

setup.kibana.host=kibana:5601
output.elasticsearch.hosts=["localhost:9200"]

I will start with the second one. With docker run command, when you are starting metricbeat, you are telling the container that it can access elastic search on localhost:9200. So when the container starts, it will access localhost on port 9200 expecting to find elasticsearch running. But, as the container is a host isolated process with its own network layer, localhost resolves to the container itself, not to your docker host machine as you are expecting.

Regarding the setup of kibana host, you should firstly understand how docker-compose works. By default, when you execute docker-compose up, a docker network is created and all services defined on yml file are added to this network. Inside this network and only, services are accessible through their service name. For your case, as defined on yml file, their names would be elasticsearch, kibana.

So in order metricbeat container to be able to communicate with elasticsearch and kibana containers, it should be added to the same docker network. This can be achieved with setting --network flag on docker run command.

Another approach would be to share docker host's network with your containers by using network mode host, but I would not recommend that.

References:

Docker compose

docker run