How do I add Dataflow memory usage to Stackdriver?

Question

Problem:

We want to understand the memory usage of our Cloud Dataflow job, preferably per worker/VM. We'd like to add this to Stackdriver, but it seems we can only add cpu usage and disk I/O metrics.

Question:

Is there a way to add Cloud Dataflow memory usage to Stackdriver?

Answer 1

As described here you can use the --experiments=enable_stackdriver_agent_metrics flag to enable Agent metrics for your Dataflow job. This will include JVM metrics such as memory/usage.