Is it possible to visualize the data on SageMaker processing job?

Question

I am working on creating a custom Sagemaker processing job that transforms my dataset. I want to plot the data matrix before and after the job i.e., visualize the job. It is possible for me to create another processing job that does this plotting. However, I prefer the job to be self-contained.

Only one option that might fit my needs comes to my mind which is monitoring through regular expressions as in plotting learning curves like here: Monitor and Analyze Training Jobs Using Amazon CloudWatch Metrics. It is pretty tedious to plot a matrix of dimensions say 10k * ~300 columns like that. So, I wonder about more native ways to do this task.

Thanks in advance

Answer 1

There is no any way by default to visualize for jobs on progress. You can publish metric you want to cloudwatch and visualize on cloudwatch or you some external applications like neptune-ai