How to pass arguments when creating kubeflow pipeline yaml file?

Question

Environment

• minikube latest (kubernets 1.21.14)
• kustomize 3.2.0
• kubectl 1.21.14
• kubeflow/manifests v1.6-branch
• KFP version 1.8.13

I want to create kubeflow pipelines file to run pipeline on GKE.
I created the pipeline according to Getting started building a pipeline.
I cannot get kubeflow pipeline yaml file by following method.

kfp.compiler.Compiler().compile(
    pipeline_func=my_pipeline,
    package_path='pipeline.yaml',
    pipeline_parameters={'url': 'https://storage.googleapis.com/ml-pipeline-playground/iris-csv-files.tar.gz'})

I got TypeError: compile() got an unexpected keyword argument 'pipeline_parameters'
From this implementation, I should be able to use pipeline_parameters argument for kfp.compiler.Compiler().compile.

Reference

• Getting started building a pipeline

• kfp.compiler package

• Option 1: Compile and then upload in UI

Answer 1

If you want to pass default pipeline parameters for your pipeline. You can do it directly in the function definition of the pipeline like this:

@dsl.pipeline(name='name')
def my_pipeline(a=1, b='default value'):
  # pipeline code
  pass

For a minimal example of a full pipeline, you can take a look at this. From my experience working with Kubeflow, a lot of the documentation are wrong or not up-to-date. Many times, you have to find a workaround.