the log file in docker of flink

Question

I'm using the flink-1.4.0 docker like:

version: '3'
services:
  jobmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6123"
    ports:
      - "8081:8081"
    command: jobmanager
    volumes:
      - /home/yeyunlong/work/flink/flink-jar:/var/log/mongodb
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

  taskmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6121"
      - "6122"
    depends_on:
      - jobmanager
    command: taskmanager
    links:
      - "jobmanager:jobmanager"
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

In the docker, the dir /opt/flink/log is empty. but use docker logs $(containerId) , I can get all of the log information.

What should I do to make the log in /opt/flink/log. Thank you!

Answer 1

I am using docker-flink images which use bin/flink-console.sh to launch the JM or TM. That script relies on settings in conf/log4j-console.properties.

I use the docker-compose.yaml below to set my own version of the Log4j properties file. From the docker-compose.yaml, I can set the file to use and from the properties file of Log4j control the output.

version: "2.1"

services:
  jobmanager:
    image: flink
    expose:
      - "6123"
    ports:
      - "8081:8081"
    command: jobmanager
    volumes:
      - /c/Users/XYZ/playground/flink/shared/log4j.properties:/opt/flink/conf/log4j-console.properties
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager
      - FLINK_ENV_JAVA_OPTS=-Dlog.file=/opt/flink/log/jobmanager.log

  taskmanager:
    image: flink
    expose:
      - "6121"
      - "6122"
    depends_on:
      - jobmanager
    command: taskmanager
    links:
      - jobmanager:jobmanager
    volumes:
      - /c/Users/XYZ/playground/flink/shared/log4j.properties:/opt/flink/conf/log4j-console.properties
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager
      - FLINK_ENV_JAVA_OPTS=-Dlog.file=/opt/flink/log/taskmanager.log