I need to select a custom profile ([Configuring your pipeline to use the custom profile] https://cloud.google.com/data-fusion/docs/how-to/running-against-existing-dataproc) for Running a pipeline against an existing Dataproc cluster. According to the type of instance I have (DEVELOPER), higher than version 6.3 the Compute config option should be enabled:
Any idea why I can't see it?
Comparison of Developer, Basic, and Enterprise editions
I’ve configured an Exiting Dataproc profile:
But when I need select this new profile in the “Compute config” option I can't see it even though the documentation indicates that in versions higher than 6.3 it should be enabled
option disabled on my instance
My Data Fusion instance:
If I understand the problem correctly, you are able to create the profile but not use in the pipelines? From the screenshot, looks like you are looking at wrong place. Here is where it can be set. https://cloud.google.com/data-fusion/docs/how-to/running-against-existing-dataproc#configuring_your_pipeline_to_use_the_custom_profile_2
