K8s Ingress to Static Assets in DigitalOcean Bucket

Question

I'm trying to use an Ingress and ExternalName Service in Kubernetes to route traffic to an external storage service (DigitalOcean Spaces) - but no matter what I try, I get some form of http error.

Things I've tried:

• https://github.com/kubernetes/ingress-nginx/pull/629#issue-116679227 (Error: 404 Not Found, nginx)
• https://github.com/kubernetes/ingress-nginx/issues/1809 (Error: 502 Bad Gateway, nginx)
• A fair bit of other tinkering which has been lost to time.

How do I configure a K8s Ingress/Service to direct ingress requests from example.com/static to a storage bucket (e.g. <zone>.digitaloceanspaces.com/<bucket-name>/<path>/<object>)?

Answer 1

It looks like some of the resources I was able to find were simply outdated. The following solution works as of Kubernetes v1.21.4.

Important Notes:

• All Ingress annotations are required:
  • kubernetes.io/ingress.class: nginx - necessary to engage Nginx ingress controller.
  • nginx.ingress.kubernetes.io/backend-protocol: HTTPS - necessary to maintain HTTPS traffic to service (this replaces /secure-backends in older versions).
  • nginx.ingress.kubernetes.io/upstream-vhost - must match service externalName, removes hostname from request path (e.g. if this is missing and being tested through localhost, will likely encounter error: "No such bucket: localhost").
  • nginx.ingress.kubernetes.io/rewrite-target - passes matched asset URL path through to service.
• The path.service.port.number in the Ingress definition must match whatever port the ExternalName service expects (443 in the case of our HTTPS traffic).

apiVersion: v1
kind: Service
metadata:
  name: do-bucket-service
spec:
  type: ExternalName
  externalName: <zone>.digitaloceanspaces.com

---

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: do-bucket-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
    nginx.ingress.kubernetes.io/rewrite-target: /<bucket>/$2
    nginx.ingress.kubernetes.io/upstream-vhost: <zone>.digitaloceanspaces.com
spec:
  rules:
  - http: 
      paths:
      - path: /path/to/static/assets(/|$)(.*)
        pathType: Prefix
        backend:
          service:
            name: do-bucket-service
            port:
              number: 443