Search code examples
jenkinskubernetesgoogle-cloud-platformdeploymentkubectl

Kubectl set image vs. apply from CI - best practice

Consider this partial k8s deployment manifest for a basic rest application called myapp

spec:
  replicas: 1
     ...
      containers:
        name: myapp
        image: us.gcr.io/my-org/myapp.3
      ...
      resources:
        limits:
          cpu: 1000m
          memory: 1.5Gi

We store incremental builds in Google Container Registry (GCR) eg (myapp.1, myapp.2, myapp.3). In the current setup, our CI system (Jenkins) does the following:

  • Docker builds new image myapp.4 and uploads to GCR
  • Runs kubectl set image myapp.4 to update the deployment.

This works well for most deploys, but what about changes to the deployment manifest itself? For example, if we changed resource > cpu to 1500m, we'd now have to manually run kubectl apply. This step needs to be automated, which brings me to my point: instead of using kubectl set image couldn't the build system itself just run kubectl apply each time? When is it appropriate to use kubectl set image vs. kubectl apply in a CI/CD pipeline?

As long as the new image were provided, wouldn't kubectly apply handle both image updates AND other config changes? If so, what are the pros/cons against just kubcetl set image?

PS - our deploys are simple and mainly rely on single replica and 100% uptime is not necessarily required.

Solution

  • With a kubectl set image, you only patch the image deployed in your deployment. To patch the other values (CPU, memory, replicas,...) you can use other commands like path or set repiclas

    The problem is that you loose the consistency with your original YAML definition. If your cluster crash and you want to recreate one, you won't have the exact same deployment because your YAML file will be outdated (the "patches" won't be known)

    With kubectl apply you overwrite the existing control plane config and you set the exact content of your YAML. It's more consistent and it's a common practice when you are in GitOps mode.

    Which one to use? All depends on what you need and what you want to achieve. I prefer the kubectl apply mode for its consistency and its replayability, but it's up to you!