I am trying to run a spring batch with remote partitioning on K8s cluster. Every thing works fine regarding the partitioning, execution etc. The same is integrated with Spring Cloud Dataflow and can see the task logs in the SCDF UI. But I am looking if there is a way to make the logs of spring batch master and worker pods to be visible in SCDF UI corresponding to each job. Any pointers in this would be really helpful.
Thanks in advance.
If the batch-job is wrapped as a Task and launched through SCDF, you should be able to gain access to the logs from the Task Executions tab.
See the "View Log" button from the Task Execution detail page. The logs should also be available to access from SCDF's RESTful APIs, as well.
Please note that the logs would only be accessible when the Task is actually running. SCDF simply interacts with K8s-API to retrieve the logs and display them in the SCDF Dashboard. The logs themselves are not persisted in SCDF, however. If you need a persistent log, you may want to pipe the logs to an APM tooling of your choice for advanced log analytics.