Search code examples
javanode.jsspringspring-cloudnetflix-eureka

Eureka detect service status

Context

We are using Spring Cloud Netflix with Eureka as the service discovery and Zuul for proxying the services and load balance them.

The microservices are implemented using NodeJS and are registered at Eureka using the NPM module eureka-js-client and a custom layer in between that handles the configuration and stuff that is generic for all microservices.

Problem

The problem is that Eureka does not recognize if one service goes down. This is a problem as we are having a development infrastructure with autodeployment that redeploys and restarts the microservices on different ports each time without restarting Eureka (and Zuul).

Therefore after a while we have ten or more instances of one microservice where only one is up but all are recognized as beeing UP.

Solution Approach

  1. I tried setting the heartbeatInterval on the client lesser but that does not help.

  2. I tried setting the renewalThresholdUpdateIntervalMs on the server lesser but that does not help either.

  3. Many more frustrating, non-helping property tries…

Question

How do I configure Eureka to evict instances or to set status to DOWN of the instances that do not send a heartbeat in a reasonable time (not 30 minutes or so)?

Code Snippets

The server itself does not contain mentionable code (just a few annotations to start the Eureka server using Spring Cloud Starter).

The configuration of the Eureka server (I have removed all non-working tries):

server:
  port: 8761

spring:
  cloud:
    client:
      hostname: localhost

eureka:
  instance:
    address: 127.0.0.1
    hostname: ${spring.cloud.client.hostname}

The client configuration that is sent to the server (using eureka-js-client):

{
    instance : {
        instanceId : `${CONFIG.instance.address}:${CONFIG.instance.name}:${CONFIG.instance.port}`,
        app : CONFIG.instance.name,
        hostName : CONFIG.instance.host,
        ipAddr : CONFIG.instance.address,
        port : {
            '$' : CONFIG.instance.port,
            '@enabled' : true
        },
        homePageUrl : `http://${CONFIG.instance.host}:${CONFIG.instance.port}/`,
        statusPageUrl : `http://${CONFIG.instance.host}:${CONFIG.instance.port}/info`,
        healthCheckUrl : `http://${CONFIG.instance.host}:${CONFIG.instance.port}/health`,
        vipAddress : CONFIG.instance.name,
        secureVipAddress : CONFIG.instance.name,
        dataCenterInfo : {
            '@class' : 'com.netflix.appinfo.InstanceInfo$DefaultDataCenterInfo',
            name : 'MyOwn'
        }
    },
    eureka : {
        host : CONFIG.eureka.host,
        port : CONFIG.eureka.port,
        servicePath : CONFIG.eureka.servicePath || '/eureka/apps/',
        healthCheckInterval : 5000
    }
}
Solution

  • after a while we have ten or more instances of one microservice where only one is up but all are recognized as beeing UP.

    Eureka has a 'self preservation' mode. Where if less than 85% of instances heartbeats are registering, it will not evict any instances. You should be able to see a warning on the eureka dashboard.