I am using Spring Cloud Netflix Ribbon in combination with Eureka in a Cloud Foundry environment.
The use case I am trying to implement is the following:
I have a running CF application named address-service with several instances spawned.
The instances are registering to Eureka by the service name address-service
I have added custom metadata to service instances using
eureka.instance.metadata-map.applicationId: ${vcap.application.application_id}
I want to use the information in Eureka's InstanceInfo (in particular the metadata and how many service instances are available) for setting a CF HTTP header "X-CF-APP-INSTANCE" as described here.
The idea is to send a Header like "X-CF-APP-INSTANCE":"appIdFromMetadata:instanceIndexCalculatedFromNoOfServiceInstances" and thus "overrule" CF's Go-Router when it comes to load balancing as described at the bottom of this issue.
I believe to set headers, I need to create a custom RibbonClient implementation - i.e. in plain Netflix terms a subclass of AbstractLoadBalancerAwareClient as described here - and override the execute() methods.
However, this does not work, as Spring Cloud Netflix Ribbon won't read the class name of my CustomRibbonClient from application.yml. It also seems Spring Cloud Netflix wraps quite a bit of classes around the plain Netflix stuff.
I tried implementing a subclass of RetryableRibbonLoadBalancingHttpClient and RibbonLoadBalancingHttpClient which are Spring classes. I tried giving their class names in application.yml using ribbon.ClientClassName but that does not work. I tried to override beans defined in Spring Cloud's HttpClientRibbonConfiguration but I cannot get it to work.
So I have two questions:
is my assumption correct that I need to create a custom Ribbon Client and that the beans defined here and here won't do the trick?
How to do it properly?
Any ideas are greatly appreciated, so thanks in advance!
Update-1
I have dug into this some more and found RibbonAutoConfiguration.
This creates a SpringClientFactory which provides a getClient() method that is only used in RibbonClientHttpRequestFactory (also declared in RibbonAutoConfiguration).
Unfortunately, RibbonClientHttpRequestFactory hard-codes the client to Netflix RestClient. And it does not seem possible to override either SpringClientFactory nor RibbonClientHttpRequestFactory beans.
I wonder if this is possible at all.
Ok, I'll answer this question myself, in case someone else may need that in the future.
Actually, I finally managed to implement it.
TLDR - the solution is here: https://github.com/TheFonz2017/Spring-Cloud-Netflix-Ribbon-CF-Routing
The solution:
The key to understanding this, is that Spring Cloud has its own LoadBalancer framework, for which Ribbon is just one possible implementation. It is also important to understand, that Ribbon is only used as a load balancer not as an HTTP client. In other words, Ribbon's ILoadBalancer instance is only used to select the service instance from the server list. Requests to the selected server instances are done by an implementation of Spring Cloud's AbstractLoadBalancingClient. When using Ribbon, these are sub-classes of RibbonLoadBalancingHttpClient and RetryableRibbonLoadBalancingHttpClient.
So, my initial approach to add an HTTP header to the requests sent by Ribbon's HTTP client did not succeed, since Ribbon's HTTP / Rest client is actually not used by Spring Cloud at all.
The solution is to implement a Spring Cloud LoadBalancerRequestTransformer which (contrary to its name) is a request interceptor.
My solution uses the following implementation:
public class CFLoadBalancerRequestTransformer implements LoadBalancerRequestTransformer {
public static final String CF_APP_GUID = "cfAppGuid";
public static final String CF_INSTANCE_INDEX = "cfInstanceIndex";
public static final String ROUTING_HEADER = "X-CF-APP-INSTANCE";
@Override
public HttpRequest transformRequest(HttpRequest request, ServiceInstance instance) {
System.out.println("Transforming Request from LoadBalancer Ribbon).");
// First: Get the service instance information from the lower Ribbon layer.
// This will include the actual service instance information as returned by Eureka.
RibbonLoadBalancerClient.RibbonServer serviceInstanceFromRibbonLoadBalancer = (RibbonLoadBalancerClient.RibbonServer) instance;
// Second: Get the the service instance from Eureka, which is encapsulated inside the Ribbon service instance wrapper.
DiscoveryEnabledServer serviceInstanceFromEurekaClient = (DiscoveryEnabledServer) serviceInstanceFromRibbonLoadBalancer.getServer();
// Finally: Get access to all the cool information that Eureka provides about the service instance (including metadata and much more).
// All of this is available for transforming the request now, if necessary.
InstanceInfo instanceInfo = serviceInstanceFromEurekaClient.getInstanceInfo();
// If it's only the instance metadata you are interested in, you can also get it without explicitly down-casting as shown above.
Map<String, String> metadata = instance.getMetadata();
System.out.println("Instance: " + instance);
dumpServiceInstanceInformation(metadata, instanceInfo);
if (metadata.containsKey(CF_APP_GUID) && metadata.containsKey(CF_INSTANCE_INDEX)) {
final String headerValue = String.format("%s:%s", metadata.get(CF_APP_GUID), metadata.get(CF_INSTANCE_INDEX));
System.out.println("Returning Request with Special Routing Header");
System.out.println("Header Value: " + headerValue);
// request.getHeaders might be immutable, so we return a wrapper that pretends to be the original request.
// and that injects an extra header.
return new CFLoadBalancerHttpRequestWrapper(request, headerValue);
}
return request;
}
/**
* Dumps metadata and InstanceInfo as JSON objects on the console.
* @param metadata the metadata (directly) retrieved from 'ServiceInstance'
* @param instanceInfo the instance info received from the (downcast) 'DiscoveryEnabledServer'
*/
private void dumpServiceInstanceInformation(Map<String, String> metadata, InstanceInfo instanceInfo) {
ObjectMapper mapper = new ObjectMapper();
String json;
try {
json = mapper.writerWithDefaultPrettyPrinter().writeValueAsString(metadata);
System.err.println("-- Metadata: " );
System.err.println(json);
json = mapper.writerWithDefaultPrettyPrinter().writeValueAsString(instanceInfo);
System.err.println("-- InstanceInfo: " );
System.err.println(json);
} catch (JsonProcessingException e) {
System.err.println(e);
}
}
/**
* Wrapper class for an HttpRequest which may only return an
* immutable list of headers. The wrapper immitates the original
* request and will return the original headers including a custom one
* added when getHeaders() is called.
*/
private class CFLoadBalancerHttpRequestWrapper implements HttpRequest {
private HttpRequest request;
private String headerValue;
CFLoadBalancerHttpRequestWrapper(HttpRequest request, String headerValue) {
this.request = request;
this.headerValue = headerValue;
}
@Override
public HttpHeaders getHeaders() {
HttpHeaders headers = new HttpHeaders();
headers.putAll(request.getHeaders());
headers.add(ROUTING_HEADER, headerValue);
return headers;
}
@Override
public String getMethodValue() {
return request.getMethodValue();
}
@Override
public URI getURI() {
return request.getURI();
}
}
}
The class is looking for the information required for setting the CF App Instance Routing header in the service instance metadata returned by Eureka.
That information is
You need to provide that in the application.yml of your service like so:
eureka:
instance:
hostname: ${vcap.application.uris[0]:localhost}
metadata-map:
# Adding information about the application GUID and app instance index to
# each instance metadata. This will be used for setting the X-CF-APP-INSTANCE header
# to instruct Go-Router where to route.
cfAppGuid: ${vcap.application.application_id}
cfInstanceIndex: ${INSTANCE_INDEX}
client:
serviceUrl:
defaultZone: https://eureka-server.<your cf domain>/eureka
Finally, you need to register the LoadBalancerRequestTransformer implementation in the Spring configuration of your service consumers (which use Ribbon under the hood):
@Bean
public LoadBalancerRequestTransformer customRequestTransformer() {
return new CFLoadBalancerRequestTransformer();
}
As a result, if you use a @LoadBalanced RestTemplate in your service consumer, the template will call Ribbon to make a choice on the service instance to send the request to, will send the request and the interceptor will inject the routing header. Go-Router will route the request to the exact instance that was specified in the routing header and not perform any additional load balancing that would interfere with Ribbon's choice.
In case a retry were necessary (against the same or one or more next instances), the interceptor would again inject the according routing header - this time for a potentially different service instance selected by Ribbon.
This allows you to use Ribbon effectively as the load balancer and de-facto disable load balancing of Go-Router, demoting it to a mere proxy. The benefit being that Ribbon is something you can influence (programmatically) whereas you have little to no influence over Go-Router.
Note: this was tested for @LoadBalanced RestTemplate's and works.
However, for @FeignClients it does not work this way.
The closest I have come to solving this for Feign is described in this post, however, the solution described there uses an interceptor that does not get access to the (Ribbon-)selected service instance, thus not allowing access to the required metadata.
Haven't found a solution so far for FeignClient.