Unable to route to external http proxy when using a kubernetes service for DNS resolution of proxy addresses

Question

I have use case where we want to route certain requests via a corporate HTTP proxy. Based on this guide I was able to configure the external access successfully. For context I've added a example ServiceEntry:

apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
  name: proxy
spec:
  addresses:
    - 10.1.1.1
    - 10.1.1.2
  exportTo:
  - .
  hosts:
  - foo.proxy # this is technically ignored when protocol is TCP
  location: MESH_EXTERNAL
  ports:
  - name: tcp
    number: 3128
    protocol: TCP

This works when we have the app automatically resolve to one of the proxy addresses above (i.e: host file entry).

In an effort to provide automatic DNS resolution I setup a a k8s Service without selectors as per the docs. In a non istio namespace, this allows me to resolve foo.proxy.default.cluster.local without the host file entries as expected e.g:

curl -v --proxy foo.default.svc.cluster.local:3128 https://blah.com

However within a istio namespace with the existing ServiceEntry (above) it fails with a 404 Not Found. The logs show:

2021-08-11T08:56:47.088919Z debug   envoy router    [C1114][S1115555414526221653] no cluster match for URL ''
2021-08-11T08:56:47.088928Z debug   envoy http  [C1114][S1115555414526221653] Sending local reply with details route_not_found

Can I get some pointers on where this might be going wrong?

Answer 1

After some trial and error the solution to make this work is as following:

1. K8s Service must have a named port

For example, I had:

apiVersion: v1
kind: Service
metadata:
  name: foo
spec:
  ports:
    - protocol: TCP
      port: 8080      

But it must include the spec.ports.[].name:

apiVersion: v1
kind: Service
metadata:
  name: foo
spec:
  ports:
    - protocol: TCP
      name: tcp # critial. alternatively prefix any name with protocol-<name>
      port: 8080      

This is documented at: https://istio.io/v1.9/docs/ops/configuration/traffic-management/protocol-selection/

For completeness here's the k8s Endpoints example:

apiVersion: v1
kind: Endpoints
metadata:
  name: foo
subsets:
  - addresses:
      - ip: 10.1.1.1
      - ip: 10.1.1.2
    ports:
      - port: 8080

The Endpoints metadata.name and Service metadata.name MUST match.

Now to the more interesting part.

1. Create a Service Entry for the VIP.

I replaced my original ServiceEntry with the following:

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: foo-forward-proxy
spec:
  hosts:
  - foo.default.svc.cluster.local
  addresses:
  - 172.20.254.32 # my foo service ClusterIP
  location: MESH_EXTERNAL
  ports:
  - number: 8080
    name: tcp
    protocol: TCP
  resolution: STATIC
  endpoints:
  - address: 10.1.1.1
  - address: 10.1.1.2

This declares the k8s Service IP 172.20.254.32 as a VIP with the name foo.default.svc.cluster.local which happens to be the Service name. All traffic destined to the VIP on port 8080 will be forwarded to the endpoints.