Search code examples
kubernetesenvironment-variables12factor

From where appeared these variables in my kubernetes container?


I am deploying simple hello-world microservice that listens on port given by following variable:

PORT = os.environ.get("TEST_SERVICE_PORT", "").strip() or "50001"

I deployed it without configuring any variables on container, and expected it to serve on default 50001 port, but instead got error

socket.gaierror: [Errno -8] Servname not supported for ai_socktype

When I logged into container and checked environment, I found out that evironment is filled with different variables (some of them belong to other services), and TEST_SERVICE_PORT variable exists and contains definitely not port:

root@test-service-697464787c-xpd6k:/opt/app/src# env | grep TEST
TEST_SERVICE_PORT_7002_TCP_ADDR=10.145.23.43
TEST_SERVICE_SERVICE_PORT_GRPC_API=7002
TEST_SERVICE_PORT_7002_TCP_PORT=7002
TEST_SERVICE_PORT=tcp://10.145.23.43:7002
TEST_SERVICE_SERVICE_HOST=10.145.23.43
TEST_SERVICE_PORT_7002_TCP=tcp://10.145.23.43:7002
TEST_SERVICE_PORT_7002_TCP_PROTO=tcp
TEST_SERVICE_SERVICE_PORT=7002

I have following questions and were not able find answers to them in documentation:

What created this variables? Could I somehow isolate container from them? Or are they set intentionally by kubernetes, and serve some purpose I don't know about? How should I name my configuration variables to avoid naming collisions? Should I use that variables istead of using services names as hostnames?

There is following documentation, but it only explains variable TEST_SERVICE_SERVICE_PORT and TEST_SERVICE_SERVICE_HOST. What TEST_SERVICE_PORT and others mean then? What adds TEST_SERVICE_SERVICE_PORT_GRPC_API?

There is also Istio and Ambassador gateway installed on cluster that I'm using.


Solution

  • Q: What created this variables?

       A: discovery-service (more at the end)

    Q: Could I somehow isolate container from them?

       A: If you want to disable that, you can set enableServiceLinks: false on your PodSpec

    Q: Or are they set intentionally by kubernetes, and serve some purpose I don't know about?

       A: No, they are there just there to give options besides DNS and names, kubernetes does not use them

    Q: How should I name my configuration variables to avoid naming collisions?

       A: Either you use enableServiceLinks: false or use a naming pattern that does not conflict with the pattern described on the docs, usually I prefer use _SVC_PORT when I need to do something like you

    Q: Should I use that variables istead of using services names as hostnames?

       A: From docs: "You can (and almost always should) set up a DNS service for your Kubernetes cluster",

    Q: There is following documentation, but it only explains variable TEST_SERVICE_SERVICE_PORT and TEST_SERVICE_SERVICE_HOST. What TEST_SERVICE_PORT and others mean then? What adds TEST_SERVICE_SERVICE_PORT_GRPC_API?

       A: You have a named port called grpc-api, in that case it is using the named instead protocol + port number. Note: I could not find any references on the docs for that, so I had dig into code


    From docs discovery-service

    When a Pod is run on a Node, the kubelet adds a set of environment variables for each active Service. ... simpler {SVCNAME}_SERVICE_HOST and {SVCNAME}_SERVICE_PORT variables, where the Service name is upper-cased and dashes are converted to underscores...

    For example, the Service "redis-master" which exposes TCP port 6379 and has been allocated cluster IP address 10.0.0.11, produces the following environment variables:

    REDIS_MASTER_SERVICE_PORT=6379 REDIS_MASTER_PORT=tcp://10.0.0.11:6379
    REDIS_MASTER_PORT_6379_TCP=tcp://10.0.0.11:6379
    REDIS_MASTER_PORT_6379_TCP_PROTO=tcp
    REDIS_MASTER_PORT_6379_TCP_PORT=6379
    REDIS_MASTER_PORT_6379_TCP_ADDR=10.0.0.11 
    

    From k8s api PodSpec/EnableServiceLinks:

    EnableServiceLinks indicates whether information about services should be injected into pod's environment variables, matching the syntax of Docker links. Optional: Defaults to true.