GKE accessing service

Question

I'm trying to connect one pod to another, but getting a connection refused error.

I only run:

1. RavenDB Server

   • Deployment which has:
     • ports:
       • containerPort:8080, protocol: TCP
       • containerPort:38888, protocol: TCP
   • Service:
     • ravendb-cluster01-service
     • clusterIP: None, ports: 8080 / 38888

2. RavenDB Client

   • Connects to ravendb-cluster01-service.staging.svc.cluster.local:8080
     • Though fails with a connection refused error

What doesn't work:

• Client cannot connect to server, connection refused

What does work:

• when accessing the client pod using interactive shell: docker -it ... -- bash,
  • I can ping the service
  • and telnet to it
• when using kubectl ... port-forward 8080:8080, I can locally enjoy the database server, so the server is running

Strange enough, when accessing the docker I'm able to connect to it, though the running script itself refuses to connect to the target pod.

It's connecting pod to pod, and tagged the target server (RavenDB) with a service, without service IP address, to resolve the domain name to the current IP address of the pod.

Any idea what I'm doing wrong?

Full config:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: ravendb-cluster01
    tier: backend
  name: ravendb-cluster01
  namespace: staging
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ravendb-cluster01
      tier: backend
  template:
    metadata:
      labels:
        app: ravendb-cluster01
        tier: backend
      name: ravendb-cluster01
      namespace: staging
    spec:
      containers:
      - env:
        - name: RAVEN_ARGS
          value: --ServerUrl=http://ravendb-cluster01-service.staging.svc.cluster.local:8080
            --ServerUrl.Tcp=tcp://ravendb-cluster01-service.staging.svc.cluster.local:38888
            --PublicServerUrl=http://localhost:8080 --PublicServerUrl.Tcp=tcp://localhost:38888
            --DataDir=/ravendb/ --Setup.Mode=None --License.Eula.Accepted=true
        image: ravendb/ravendb-nightly:4.0.6-nightly-20180720-0400-ubuntu.16.04-x64
        name: ravendb
        ports:
        - containerPort: 8080
          name: http
          protocol: TCP
        - containerPort: 38888
          name: tcp
          protocol: TCP
        resources:
          limits:
            memory: 26000Mi
          requests:
            memory: 26000Mi
        volumeMounts:
        - mountPath: /ravendb/
          name: ravendb-cluster01-storage
      volumes:
      - gcePersistentDisk:
          fsType: ext4
          pdName: ravendb-cluster01-storage
        name: ravendb-cluster01-storage
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: ravendb-cluster01-service
    tier: backend
  name: ravendb-cluster01-service
  namespace: staging
spec:
  clusterIP: None
  ports:
  - name: http
    port: 8080
    protocol: TCP
    targetPort: 8080
  - name: tcp
    port: 38888
    protocol: TCP
    targetPort: 38888
  selector:
    app: ravendb-cluster01
    tier: backend
  sessionAffinity: None
  type: ClusterIP

Answer 1

The issue appears to be your PublicServerUrl setting.

--PublicServerUrl=http://localhost:8080 --PublicServerUrl.Tcp=tcp://localhost:38888

As per the RavenDB documentation:

Set the URL to be accessible by clients and other nodes, regardless of which IP is used to access the server internally. This is useful when using a secured connection via https URL, or behind a proxy server.

You either need to configure this to be the service name, or remove the option entirely. After reviewing the docs for ServerUrl I would personally recommend updating your args to be something like this:

value: --ServerUrl=http://0.0.0.0:8080
            --ServerUrl.Tcp=tcp://0.0.0.0:38888
            --PublicServerUrl=http://ravendb-cluster01-service.staging.svc.cluster.local:8080 --PublicServerUrl.Tcp=tcp://ravendb-cluster01-service.staging.svc.cluster.local:38888
            --DataDir=/ravendb/ --Setup.Mode=None --License.Eula.Accepted=true

You want the ServerUrl to be listening on all ports ideally, so setting to 0.0.0.0 makes sense for the PublicUrl.

The reason it works with both port-forward and from the local docker container is probably because RavenDB is listening on the loopback device, and both those methods of connection give you a local process inside the container, so the loopback device is accessible.