Krakend can't get running services from consul (Docker)

Question

I am using Consul as SD and while I am running multiple instances I want make gateway recognize them all to load balance. In case of hardcoded host for Krakend config everything works fine

I was going step by step with this: https://www.krakend.io/docs/backends/service-discovery/

Sending

dig @127.0.0.1 -p 8600 user-ms.service.consul SRV gives me response as well (on consul logs a also see that I've made a request)

But when I am making request via krakenD I am getting "no hosts available" and according to logs request wasn't send to consul

Here is a docker-compose file:

services:
  postgres:
     image: postgres:13
     restart: 'always'
     environment:
       - POSTGRES_DB=user-db
       - POSTGRES_PASSWORD=password
       - POSTGRES_USER=user
     ports:
       - "5432:5432"

 consul:
   image: consul:latest
   restart: 'always'

   ports:
     - '8500:8500'
     - '8600:8600/tcp'
     - '8600:8600/udp'

 krakend_gateway:
   image: devopsfaith/krakend:2
   volumes:
     - ./krakend.json:/krakend.json
   ports:
     - "1234:1234"
     - "8080:8080"
     - "8090:8090"

   command: [ "run", "-d", "-c", "/krakend.json" ]

 user-ms:
   build: user-ms/
   platform: linux/arm64
   restart: 'always'
   depends_on: [consul-server , krakend_gateway]
   ports:
     - '8082'

And pretty simple Krakend.json config file

{
  "version": 3,
  "timeout": "3000ms",
  "cache_ttl": "300s",
  "host" : "localhost:8080",
  "endpoints": [
    {
      "endpoint": "/users",
      "backend": [
        {
          "url_pattern": "/api/v1/users",
          "sd" : "dns",
          "host" : [
            "user-ms.service.consul.srv"
          ],
          "disable_host_sanitize": true
        }
      ]
    }
  ]
}

I've tried a lot of thing , but still get the same response , will be glad for any help with this. Thanks

Answer 1

when I am making request via krakenD I am getting "no hosts available" and according to logs request wasn't send to consul

The reason your request is failing is because Consul's DNS server listens on the non-standard port of 8600. Kraken does not appear to support configuring a custom DNS IP and non-standard port. It uses the underlying host's configured nameservers in /etc/resolv.conf resolve SRV requests, and expects these IPs to be reachable over port 53.

The following is a Docker compose file that includes the following changes to allow Kraken to resolve DNS against Consul.

# docker-compose.yaml
---
services:
  postgres:
    image: postgres:13
    restart: 'always'
    environment:
      - POSTGRES_DB=user-db
      - POSTGRES_PASSWORD=password
      - POSTGRES_USER=user
    ports:
      - "5432:5432"
    networks:
      - vpcbr

  consul:
    image: consul:latest
    restart: 'always'
    environment:
      CONSUL_LOCAL_CONFIG: |
        {
          "recursors": [
            "8.8.8.8",
            "8.8.4.4"
          ],
          "dns_config": {
            "recursor_strategy": "random"
          },
          "ports": {
            "dns": 53
          },
          "services": [
            {
              "name": "user-ms",
              "address": "192.0.2.20",
              "port": 8082
            }
          ]
        }
    networks:
      vpcbr:
        ipv4_address: 192.0.2.10
    ports:
      - '8500:8500'
      - '8600:53/tcp'
      - '8600:53/udp'

  krakend_gateway:
    image: devopsfaith/krakend:2
    command: [ "run", "-d", "-c", "/krakend.json" ]
    dns: 192.0.2.10
    volumes:
      - ./krakend.json:/krakend.json
    ports:
      - "1234:1234"
      - "8080:8080"
      - "8090:8090"
    networks:
      - vpcbr

  user-ms:
    build: user-ms/
    platform: linux/arm64
    restart: 'always'
    depends_on: [consul, krakend_gateway]
    networks:
      vpcbr:
        ipv4_address: 192.0.2.20
    ports:
      - '8082'

networks:
  vpcbr:
    driver: bridge
    ipam:
      config:
      - subnet: 192.0.2.0/24

Docker compose changes

1. The containers have been deployed onto a separate private network, vpcbr, with the IP range of 192.0.2.0/24. This is so that a static IP can be assigned to the Consul container.
2. The consul container has been issued a static IP of 192.0.2.10 from the vpcbr network.
3. The remaining containers have been configured to obtain a dynamic IP from vpcbr.
4. The krakend_gateway container has been configured to use 192.0.2.10 (Consul) for upstream DNS resolution.

Consul changes

1. Consul has been configured with a static registration for the user-ms service. Alternatively, you can use a program like Registrator to dynamically register containers from Docker into Consul.
2. Consul has been updated to listen on port 53 instead of 8600.
3. Consul has also been configured with a set of upstream DNS recursors so that it can resolve DNS queries for non .consul TLDs.

Kraken changes

1. The example DNS hostname in Kraken's documentation is incorrect for integrating with Consul. The .srv suffix should be removed as Consul only handles queries for records under the .consul top-level domain.

{
  "version": 3,
  "timeout": "3000ms",
  "cache_ttl": "300s",
  "host": "localhost:8080",
  "endpoints": [
    {
      "endpoint": "/users",
      "backend": [
        {
          "url_pattern": "/api/v1/users",
          "sd": "dns",
          "host": [
            "user-ms.service.consul"
          ],
          "disable_host_sanitize": true
        }
      ]
    }
  ]
}

With this configuration, I can see that Kraken successfully resolves the user-ms.service.consul domain against Consul DNS.