Search code examples
bashubuntudockerlogstashboot2docker

Logstash in Docker - config file not found when mounted though a volume

EDIT: It seems to be a boot2docker thing... works fine in an ubuntu vm running docker.

I'm trying to fire up logstash in a Docker container but when I mount the config file as a volume, it doesn't seem to find it. Interestingly, if I start up the container with bash, I can see the config file in there and running the same command as what docker would, it works.

Docker File

FROM ubuntu:14.04
MAINTAINER cvallance

RUN apt-get update
RUN apt-get -yqq install openjdk-7-jre
RUN apt-get -yqq install curl
RUN mkdir /opt/logstash \
  && cd /opt/logstash \
  && curl -O https://download.elastic.co/logstash/logstash/logstash-1.4.2.tar.gz \
  && tar zxvf logstash-1.4.2.tar.gz

CMD ["/opt/logstash/logstash-1.4.2/bin/logstash", "agent", "--verbose", "-f", "/etc/logstash/sample.conf"]

Config file located at $(pwd)/config/sample.conf

input {
  stdin { }
}

output {
  stdout {
    codec => rubydebug
  }
}

Docker build command:

docker build -t cvallance/logstash .

Docker run command:

docker run -ti -v $(pwd)/config:/etc/logstash cvallance/logstash

Error:

Error: No config files found:  /etc/logstash/sample.conf
Can you make sure this path is a logstash config file?

BUT if I run the same command inside the container from a bash session like so...

docker run -ti -v $(pwd)/config:/etc/logstash cvallance/logstash bash
...
root@d3fd885903dd:/# /opt/logstash/logstash-1.4.2/bin/logstash agent --verbose -f /etc/logstash/sample.conf

Everything works as expected. I.e. input and output

testing
{
       "message" => "testing",
      "@version" => "1",
    "@timestamp" => "2015-04-10T00:06:33.878Z",
          "host" => "d3fd885903dd"
}
Solution

  • If it is a "boot2docker" thing, remember that is offers you a Linux host based on Tiny core, without any persistence (except /var/lib/docker).

    What I do is make sure the Oracle_VM_VirtualBox_Extension_Pack is installed, and get my /c/Users/<yourLogin> automatically shared in my boot2docker ssh session.
    Whatever I need to keep is written there (/c/Users/<yourLogin>/...), not anywhere else.
    If you have to mount a host directory as a data volume, you should mount a folder from /c/Users/<yourLogin>/....

    The other option (which does persists) is to define a Data Volume Container, which will persists data in /var/lib/docker/volumes/conf.json, and (for the actual files) in /var/lib/docker/vfs/xxx.
    That is the preferred best-practice as it doesn't link your data container to a specific host platform (here '/c/Users/<yourLogin> actually depends on the VM host, on Windows! Not very portable.)