Run Filebeat in docker as IoT Edge module

Question

I would like to run Filebeat as Docker container in Azure IoT Edge. I would like Filebeat to get logs from others running containers.

I'm already able to run filebeat as Docker container, from the documentation (https://www.elastic.co/guide/en/beats/filebeat/6.8/running-on-docker.html#_volume_mounted_configuration)

docker run -d \
  --name=filebeat \
  --user=root \
  --volume="$(pwd)/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro" \
  --volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" \
  --volume="/var/run/docker.sock:/var/run/docker.sock:ro" \
  docker.elastic.co/beats/filebeat:6.8.3 filebeat -e -strict.perms=false

With this command and with the correct filebeat.yml file I'm able to collect logs for every running containers on my device.

Now I would like to deploy this configuration as Azure IoT Edge Modules.

I created a docker image having the filebeat.yml file included with the following Dockerfile:

FROM docker.elastic.co/beats/filebeat:6.8.3
COPY filebeat.yml /usr/share/filebeat/filebeat.yml
USER root
RUN chmod go-w /usr/share/filebeat/filebeat.yml
RUN chown root:filebeat /usr/share/filebeat/filebeat.yml
USER filebeat

From documentation: https://www.elastic.co/guide/en/beats/filebeat/6.8/running-on-docker.html#_custom_image_configuration

I tested this Dockerfile by running locally

docker build -t filebeat .

and

docker run -d \
  --name=filebeat \
  --user=root \
  --volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" \
  --volume="/var/run/docker.sock:/var/run/docker.sock:ro" \
  filebeat:latest filebeat -e -strict.perms=false

This works fine, logs from other containers are collected as they should.

Now my question is :

• In Azure IoT Edge, how can I mount volumes to access others Docker containers running on the devices, like it's done with

--volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" \ 
--volume="/var/run/docker.sock:/var/run/docker.sock:ro"

in order to collect logs?

From this other SO post (Mount path to Azure IoT Edge module) in the Azure IoT Edge portal I tried the following:

  "HostConfig": {
    "Mounts": [
      {
        "Target": "/var/lib/docker/containers",
        "Source": "/var/lib/docker/containers",
        "Type": "volume",
        "ReadOnly: true
      },
      {
        "Target": "/var/run/docker.sock",
        "Source": "/var/run/docker.sock",
        "Type": "volume",
        "ReadOnly: true
      }
    ]
  }
}

But when I deploy this module I have the following error:

2019-11-25T10:09:41Z [WARN] - Could not create module FilebeatAgent
2019-11-25T10:09:41Z [WARN] -         caused by: create /var/lib/docker/containers: "/var/lib/docker/containers" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path

I don't understand this error. How can I specify a path using only [a-zA-Z0-9][a-zA-Z0-9_.-] ?

Thanks for your help.

EDIT

In the Azure IoT Edge portal, createOptions json:

{
  "HostConfig": {
    "Binds": [
      "/var/lib/docker/containers:/var/lib/docker/containers",
      "/var/run/docker.sock:/var/run/docker.sock"
    ]
  }
}

Answer 1

There is an article that describes how to mount storage from the host here: https://learn.microsoft.com/en-us/azure/iot-edge/how-to-access-host-storage-from-module