Search code examples
kubernetesargo-workflows

How does an Argo Workflow carry out control flow with Kubernetes?


If desperate, one can consider Argo Workflows as a programming language implemented in YAML and using Kubernetes as a back-end.

  • A procedure can be defined using steps:
  • Functions are Templates with arguments coming in two flavours:
    • Parameters, which are strings
    • Artifacts, which are files shared by some tool, such as S3 or NFS
  • There is flow control
    • Conditionals are implemented by when:
    • Iterators are implemented by withSequence: and withItems:
    • Recursion is possible by Templates calling themselves

The templates map somewhat directly onto Kubernetes YAML specs. Parameters appear to be shared via annotations and artifacts are shared via native Kubernetes functionality.

How is the flow-control implemented? What features of Kubernetes does Argo use to accomplish this? Does it have something to do with the Kubernetes Control Plane?


Solution

  • What are the relevant Kubernetes components?

    In Linux, a running unit of code is a process. Processes get loaded from binary executable files stored in the filesystem. Traditionally, such code is actually split between multiple files (including libraries, which are each shared by multiple applications) for modularity and efficiency.

    Containerisation is the practice of bundling all the files touched by a software application (binary executables, binary libraries, configuration data files, etc) into a single container image. This is done for portability, meaning that there is no installation procedure needed and no potential incompatibility (like untested or conflicting variations of the same library) between separate containers.

    Kubernetes is the standard open source scheme for running containerised processes on clusters of computers.

    The compute nodes in a Kubernetes cluster each contain software for running containers and coordinating with the control plane. The control plane includes a mental model (stored in a database but expressible as YAML Kubernetes resource manifests) of the entire desired state of the cluster and an API server to let utilities interact with this model. Kubernetes has various controller components, which monitor and react so as to automatically maintain the cluster in its desired state.

    Some controllers are internal to the control plane, such as the scheduler that assigns desired pods (i.e., coupled groups of containers) to particular nodes. Some controllers are deployed (like any other workload) to run from a container someplace in the cluster, such as the ArgoWorkflows workflow controller.

    What does the ArgoWorkflows interface do?

    The argo CLI utility (or alternatively the ArgoWorkflows web UI) communicates with the Kubernetes API server to create and modify the "workflow" object (which exist in the mental model). For example, argo submit will create a new workflow object, and argo stop will modify a field (expressing the new desired status) of an existing workflow object.

    Note that from the perspective of the Kubernetes control plane, the workflow object is just a chunk of arbitrary metadata to be stored.* It has no semantic meaning, although the API server does get given a schema (CRD) for validating the syntax (or rather the field layout) of workflows. (*Unfortunately ArgoWorkflows abuses the Kubernetes model somewhat by storing large amounts of data, such as thousands of subtask summaries, as a compressed blob in one of the workflow status fields. This breaks some Kubernetes tools. ArgoWorkflows also resorts to having a separate database and API server, for archiving inactive workflows, relieving some pressure from the Kubernetes control plane.)

    What does the ArgoWorkflows controller do?

    The workflow controller is what implements the flow control specified by the workflow manifest.

    The workflow controller is the only component that understands the semantic meaning of the fields in a workflow. It updates status fields of the workflow object to keep track of what phase the workflow is up to, decides what the cluster should do next, and requests new pods with appropriate configuration to carry out those steps.

    What does the Emissary do?

    When creating a worker pod, the workflow controller bundles a sidecar into the specification, which will inject a replacement top level executable (argoexec) into the container to oversee the intended worker process.

    The main purpose of this emissary is to capture the outputs when the worker process finishes its subtask. It could potentially also decouple the subtask from the pod lifecycle.