Redirect task log of Spring Cloud Data Flow

Question

Is there any way to locate, copy, or manipulate logs of task execution, in SCDF, in local?

I'm currently seeing logs whenever I execute batch (or not batch) task in cmdline of shell where I've started dataflow server locally. In both CentOS 7 and Windows 10, it says that it located their stdout/stderr logs inside

/tmp (temp in windows)/[SOME_NUMBER_I_DON'T_KNOW]/${task_name_i_defined}_${SOME_HEX_CODE_RELATED_TO_TASK_EXECUTION_ID}

I want to use that information whenever I need.

Passing properties to dataflow jar doesn't work. It just creates a file, writes that file over and over at each task execution, unlike storing each task execution at different folder.

Modifying properties like loggig.file.path at task lauching configurations doesn't work, either. Only stdout of task is made with the name of 'spring.log', at specific location i designated. Behavior is same as above case.

Spring Cloud Data Flow Task logs

I looked at this answer, but it does not work, either...

I know there are a lot of parameters that I could pass to dataflow or tasks. I don't think none of them could satisfy this condition. Please enlighten me.

Answer 1

The only configuration property available to effect the log location is the working-directories-root deployer property.

Because it is a deployer property, it can not simply be set as spring.cloud.deployer.local.working-directories-root.

• It can set at task launch time and prefixed w/ deployer.*.local (details).

• It can also be configured globally via the "task platform" properties (details).

When configured at the platform level, it can be done in yml:

spring:
  cloud:
    dataflow:
      task:
        platform:
          local:
            accounts:
              default:
                working-directories-root: /Users/foo/logz

or via an env var:

SPRING_CLOUD_DATAFLOW_TASK_PLATFORM_LOCAL_ACCOUNTS_DEFAULT_WORKING_DIRECTORIES_ROOT=/Users/foo/logz

Details

The STDOUT log location is created at <task-work-dir>/stdout.log (details).

The <task-work-dir> is defined as:

<working-directories-root> / System.nanoTime() / <taskLaunchId> (details)

The <working-directories-root> is the value of the working-directories-root local deployer property or the "java.io.tmpdir" system property when the local deployer property is not specified.