skaffold + helm integration

I am trying to play around with skaffold and helm for an internal project. Before I ask the actual question I have, here is the skaffold.yaml:

apiVersion: skaffold/v2beta17
kind: Config
build:
  tagPolicy:
    gitCommit: {}
  artifacts:
    - image: my-image
      custom:
        buildCommand: ./build.sh
  local:
    useDockerCLI: false
    useBuildkit: false
    push: false


deploy:
  helm:
    releases:
      - name: config-map-it
        chartPath: src/main/helm
        artifactOverrides:
          # skaffold will override this with a different tag
          image: my-image
        valuesFiles:
          - src/main/helm/values.yaml
        wait: true
        setValues:
          namespace: my-namespace

As you might notice I have a custom way to build the image. It's actually multiple steps, but eventually it does something like this:

#!/usr/bin/env bash

# build jar only, no tests, no chart
gradle clean build -x test -x helmChartBuild --quiet

#  ... more steps here    
docker build --build-arg JAR_FILE='build/libs/*.jar'  -t ${IMAGE} .

# load docker image into kind 
kind load docker-image ${IMAGE} --name my-namespace

Specifically, notice ${IMAGE}. I need this, because I am going to invoke :

skaffold build
skaffold deploy

(and yes, this is how I need it for other unrelated reasons)

So, I invoke skaffold build first. I get an image like:

myimage/a3eadcc-dirty

The next step : skaffold deploy will fail with:

no tag provided for image [my-image]

I understand why and I slightly change the commands now:

skaffold build --file-output=tag.json
kind load docker-image ${IMAGE} --name my-namespace
skaffold deploy -a tag.json

Unfortunately, this will not work either.

The problem is that ${IMAGE} is going to be myimage/a3eadcc-dirty, while tag.json is going to contain the sha of the image as the tag, so deploy will fail because such an image can not be found.

The solution in my case would not be complicated to be fair. Just parse ${IMAGE}, find out the tag:

export MY_TAG=...

and use that with skaffold deploy --image...

The question is, is this the way?

Solution

So you seem to have a couple of issue: how do you determine the image tags? How do you test the images and manifests?

You can use skaffold build --tag xxx to override the tag policy and build the images with a fixed tag name.

Rather than try to run tests as part of your build, which will greatly slow down your normal development loop, use Skaffold's testing phase. You can run image tests with the skaffold test (docs). The custom test allows running arbitrary commands. You can run manifest validations as part of this test phase.

With Helm in the picture, you need to render (or hydrate) the manifests before you can validate them. You could skaffold build --file-output images.json && skaffold render --build-artifacts=images.json > manifests.yaml && validate manifests.yaml.

If your manifest validator expects to retrieve the images, but cannot retrieve images from the local Docker daemon, then you could instead build to a private registry. You can create a temp registry with the registry:2 image; the temp registry also helps ensure that the images don't escape.