Search code examples
kubeflow-pipelinesgoogle-cloud-vertex-ai

how to concatenate the OutputPathPlaceholder with a string with Kubeflow pipelines?


I am using Kubeflow pipelines (KFP) with GCP Vertex AI pipelines. I am using kfp==1.8.5 (kfp SDK) and google-cloud-pipeline-components==0.1.7. Not sure if I can find which version of Kubeflow is used on GCP.

I am bulding a component (yaml) using python inspired form this Github issue. I am defining an output like:

outputs=[(OutputSpec(name='drt_model', type='Model'))]

This will be a base output directory to store few artifacts on Cloud Storage like model checkpoints and model.

I would to keep one base output directory but add sub directories depending of the artifact:

  • <output_dir_base>/model
  • <output_dir_base>/checkpoints
  • <output_dir_base>/tensorboard

but I didn't find how to concatenate the OutputPathPlaceholder('drt_model') with a string like '/model'.

How can append extra folder structure like /model or /tensorboard to the OutputPathPlaceholder that KFP will set during run time ?


Solution

  • I didn't realized in the first place that ConcatPlaceholder accept both Artifact and string. This is exactly what I wanted to achieve:

    ConcatPlaceholder([OutputPathPlaceholder('drt_model'), '/model'])