I have created a custom Dataflow Template that uses a few mandatory and optional parameters. I am able to create a Template but when I open it in Dataflow UI, it does not show my declared required parameters.
I got one annotation @Validation.Required
but that is also not working. Is there anything I am missing?
My Options Class is
package org.example;
import org.apache.beam.sdk.extensions.gcp.options.GcpOptions;
import org.apache.beam.sdk.options.Default;
import org.apache.beam.sdk.options.Description;
import org.apache.beam.sdk.options.Validation;
import org.apache.beam.sdk.options.ValueProvider;
public interface TestOptions extends GcpOptions {
@Description("Project Id")
@Default.String("Project Id")
ValueProvider<String> getProjectId();
void setProjectId(ValueProvider<String> value);
@Validation.Required
@Description("Path to the input file")
@Default.String("gs://nikhil-test-bucket/Test/input.txt")
ValueProvider<String> getInputFile();
void setInputFile(ValueProvider<String> value);
@Validation.Required
@Description("Path to the output file")
@Default.String("gs://nikhil-test-bucket/Test/output.txt")
ValueProvider<String> getOutputFile();
void setOutputFile(ValueProvider<String> value);
@Description("Path to the Config file")
@Default.String("gs://nikhil-test-bucket/Test/config.txt")
ValueProvider<String> getConfigFile();
void setConfigFile(ValueProvider<String> value);
}
The Dataflow UI reads the _metadata
file (also called spec) for the list of parameters, not the actual source code.
In the metadata parameter, you can specify "isOptional": true
for any parameter.