Search code examples
pythonprefect

How to cache/target tasks with the same name in a Flow with prefect?


I am trying to find a target pattern or cache config to differentiate between tasks with the same name in a flow.

enter image description here

As highlighted from the diagram above only one of the tasks gets cached and the other get overwritten. I tried using task-slug but to no avail.

@task(
    name="process_resource-{task_slug}",
    log_stdout=True,
    target=task_target
    )

Thanks in advance


Solution

  • It looks like you are attempting to format the task name instead of the target. (task names are not template-able strings).

    The following snippet is probably what you want:

    @task(name="process_resource", log_stdout=True, target="{task_name}-{task_slug}")