Search code examples
dockerapache-flink

There is no Logs and Stdout in Flink WebUI

Recently I have started a cluster of flink with Docker-compose.Here is my file:

version: "2.1"
services:
  jobmanager:
    image: flink:1.9.2-scala_2.11
    expose:
      - "6123"
    ports:
      - "8081:8081"
    command: jobmanager
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

  taskmanager:
    image: flink:1.9.2-scala_2.11
    expose:
      - "6121"
      - "6122"
    depends_on:
      - jobmanager
    command: taskmanager
    links:
      - "jobmanager:jobmanager"
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

I started it with "docker-compose up -d --scale taskmanager=3" to assign 3 taskmanagers. I have passed the WordCount Demo and when I input some words in the bash there is no error occurs.

However, I just cannot see the logs and stdout from the WebUI WebUI

Solution

  • In containerized deployments, Flink's logs are sent to stdout/stderr rather than to log files, and infrastructure is available for log collection. With docker-compose, you can use docker-compose logs -f to follow all of the logs, docker-compose logs -f [SERVICE] to follow the logs for one service, etc.

    Because the logs are not being written to the log files, Flink's web UI is unable to display them.

    For what it's worth, there has been some discussion on the [email protected] mailing list about writing the logs to both stdout/stderr and the log files.