skaffold does not reload golang code in minikube

I've been experimenting with skaffold with a local minikube installation. It's a nice to be able to develop your project on something that is as close as possible to production.

If I use the getting-started example provided on skaffold github repo, everything works just fine, my IDE (intellij idea) stops on the breakpoints and when I modify my code, the changes are reflected instantly.

Now on my personal project which is a bit more complicated than a simple main.go file, things don't work as expected. The IDE stops on the breakpoint but hot code reload are not happening even though I see in the console that skaffold detected the changes made on that particular file but unfortunately the changes are not reflected/applied.

A docker file is used to build an image, the docker file is the following

FROM golang:1.14 AS builder

WORKDIR /app

COPY go.mod go.sum ./
RUN go mod download
COPY . .

RUN CGO_ENABLED=0 go build -o /app.o ./cmd/shortener/shortener.go

FROM alpine:3.12
COPY --from=builder /app.o ./
COPY --from=builder /app ./
EXPOSE 3000
ENV GOTRACEBACK=all
CMD ["./app.o"]

On kubernetes side, I'm creating a deployment and a service as follows:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: url-shortener-deployment
spec:
  selector:
    matchLabels:
      app: url-shortener
  template:
    metadata:
      labels:
        app: url-shortener
    spec:
      containers:
        - name: url-shortener
          image: url_shortener
          ports:
            - containerPort: 3000
---
apiVersion: v1
kind: Service
metadata:
  name: url-shortener-service
spec:
  selector:
    app: url-shortener
  ports:
    - port: 3000
      nodePort: 30000
  type: NodePort

As for skaffold, here's the skaffold.yaml file:

apiVersion: skaffold/v2beta5
kind: Config
metadata:
  name: url-shortener
build:
  artifacts:
    - image: url_shortener
      context: shortener
      docker:
        dockerfile: build/docker/Dockerfile.dev
        noCache: false
deploy:
  kubectl:
    manifests:
      - stack/mongo/mongo.yaml
      - shortener/deployments/kubernetes/shortener.yaml

I've enabled verbose logging and I notice this in the output whenever I save (CTRL+S) a source code file.

time="2020-07-05T22:51:08+02:00" level=debug msg="Found dependencies for dockerfile: [{go.mod /app true} {go.sum /app true} {. /app true}]"
time="2020-07-05T22:51:08+02:00" level=info msg="files modified: [shortener/internal/handler/rest/rest.go]"

I'm assuming that this means that the change has been detected.

breakpoints works correctly in the IDE but code swap in kubernetes don't seem to be happening

Solution

The debug functionality deliberately disables Skaffold's file-watching, which rebuilds and redeploys containers on file change. The redeploy causes existing containers to be terminated, which tears down any ongoing debug sessions. It's really disorienting and aggravating to have your carefully-constructed debug session be torn down because you accidentally saved a change to a comment! 😫

But we're looking at how to better support this more iterative debugging within Cloud Code.

If you're using Skaffold directly, we recently added the ability to re-enable file-watching via skaffold debug --auto-build --auto-deploy (present in v1.12).