I'm currently working on a microservices application for my internship using Consul for service discovery and feign clients for communicating between the services. When we started working on the existing project which already was built using microservices, we upgraded Spring boot to 2.4.3 & cloud to 2020.0.1, so that we could make use of Java 15 to use records instead of normal classes for dtos. The problem we have now is that, whenever we make a call to a composite service, that will try to retrieve data from multiple services (for example users and teams service), that we get the following stacktrace:
21-05-25 Tue 09:15:13.368 ERROR request UT005023: Exception handling request to /new/create/team
org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.springframework.retry.RetryException: Could not recover; nested exception is java.lang.AbstractMethodError: Receiver class org.springframework.cloud.netflix.ribbon.RibbonLoadBalancerClient does not define or inherit an implementation of the resolved method 'abstract org.springframework.cloud.client.ServiceInstance choose(java.lang.String, org.springframework.cloud.client.loadbalancer.Request)' of interface org.springframework.cloud.client.loadbalancer.ServiceInstanceChooser.
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014) ~[spring-webmvc-5.3.4.jar:5.3.4]
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909) ~[spring-webmvc-5.3.4.jar:5.3.4]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:517) ~[jakarta.servlet-api-4.0.4.jar:4.0.4]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) ~[spring-webmvc-5.3.4.jar:5.3.4]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:584) ~[jakarta.servlet-api-4.0.4.jar:4.0.4]
at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-5.3.4.jar:5.3.4]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.4.jar:5.3.4]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-5.3.4.jar:5.3.4]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.4.jar:5.3.4]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:93) ~[spring-boot-actuator-2.4.3.jar:2.4.3]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.4.jar:5.3.4]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-5.3.4.jar:5.3.4]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.4.jar:5.3.4]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:117) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) ~[undertow-core-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) ~[undertow-core-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) ~[undertow-core-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) ~[undertow-core-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) ~[undertow-core-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.SendErrorPageHandler.handleRequest(SendErrorPageHandler.java:52) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) ~[undertow-core-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:269) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:78) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:133) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:130) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:249) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:78) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:99) ~[undertow-servlet-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:387) ~[undertow-core-2.2.4.Final.jar:2.2.4.Final]
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:841) ~[undertow-core-2.2.4.Final.jar:2.2.4.Final]
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) ~[jboss-threads-3.1.0.Final.jar:3.1.0.Final]
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2019) ~[jboss-threads-3.1.0.Final.jar:3.1.0.Final]
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1558) ~[jboss-threads-3.1.0.Final.jar:3.1.0.Final]
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1449) ~[jboss-threads-3.1.0.Final.jar:3.1.0.Final]
at java.base/java.lang.Thread.run(Thread.java:832) ~[na:na]
Caused by: org.springframework.retry.RetryException: Could not recover; nested exception is java.lang.AbstractMethodError: Receiver class org.springframework.cloud.netflix.ribbon.RibbonLoadBalancerClient does not define or inherit an implementation of the resolved method 'abstract org.springframework.cloud.client.ServiceInstance choose(java.lang.String, org.springframework.cloud.client.loadbalancer.Request)' of interface org.springframework.cloud.client.loadbalancer.ServiceInstanceChooser.
at org.springframework.cloud.client.loadbalancer.LoadBalancedRecoveryCallback.recover(LoadBalancedRecoveryCallback.java:56) ~[spring-cloud-commons-3.0.1.jar:3.0.1]
at org.springframework.retry.support.RetryTemplate.handleRetryExhausted(RetryTemplate.java:539) ~[spring-retry-1.3.1.jar:na]
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:387) ~[spring-retry-1.3.1.jar:na]
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:225) ~[spring-retry-1.3.1.jar:na]
at org.springframework.cloud.openfeign.loadbalancer.RetryableFeignBlockingLoadBalancerClient.execute(RetryableFeignBlockingLoadBalancerClient.java:103) ~[spring-cloud-openfeign-core-3.0.1.jar:3.0.1]
at feign.SynchronousMethodHandler.executeAndDecode(SynchronousMethodHandler.java:119) ~[feign-core-10.10.1.jar:na]
at feign.SynchronousMethodHandler.invoke(SynchronousMethodHandler.java:89) ~[feign-core-10.10.1.jar:na]
at feign.ReflectiveFeign$FeignInvocationHandler.invoke(ReflectiveFeign.java:100) ~[feign-core-10.10.1.jar:na]
at com.sun.proxy.$Proxy131.createParticipant(Unknown Source) ~[na:na]
at be.jarchitects.be.usercomposite.controllers.RegistrationController.createTeamAndUser(RegistrationController.java:25) ~[classes/:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:564) ~[na:na]
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:197) ~[spring-web-5.3.4.jar:5.3.4]
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:141) ~[spring-web-5.3.4.jar:5.3.4]
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106) ~[spring-webmvc-5.3.4.jar:5.3.4]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:894) ~[spring-webmvc-5.3.4.jar:5.3.4]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808) ~[spring-webmvc-5.3.4.jar:5.3.4]
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.3.4.jar:5.3.4]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1060) ~[spring-webmvc-5.3.4.jar:5.3.4]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:962) ~[spring-webmvc-5.3.4.jar:5.3.4]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) ~[spring-webmvc-5.3.4.jar:5.3.4]
... 54 common frames omitted
Caused by: java.lang.AbstractMethodError: Receiver class org.springframework.cloud.netflix.ribbon.RibbonLoadBalancerClient does not define or inherit an implementation of the resolved method 'abstract org.springframework.cloud.client.ServiceInstance choose(java.lang.String, org.springframework.cloud.client.loadbalancer.Request)' of interface org.springframework.cloud.client.loadbalancer.ServiceInstanceChooser.
at org.springframework.cloud.openfeign.loadbalancer.RetryableFeignBlockingLoadBalancerClient.lambda$execute$2(RetryableFeignBlockingLoadBalancerClient.java:126) ~[spring-cloud-openfeign-core-3.0.1.jar:3.0.1]
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:329) ~[spring-retry-1.3.1.jar:na]
... 74 common frames omitted
Feign uses ribbon for loadbalancing by default, but Consul already load balances incoming requests.
We disabled ribbon using spring.cloud.consul.loadbalancer.ribbon.enabled: false in our configuration service, but I've read some posts that people using the same spring versions, have a similar problem and that the ribbon loadbalancer doesn't seem to disable. It was also stated on the github of spring boot under breaking changes that some netflix modules have been removed, so I assume this is related to the issue: https://github.com/spring-cloud/spring-cloud-release/wiki/Spring-Cloud-2020.0-Release-Notes#known-issues
The configuration in the configuration service is being properly provided, it says so in the logs and I also tested it with some random port and they all seem to apply it properly. Putting the property in the bootstrap yaml isn't working either.
Spring cloud loadbalancer is appearantly the new replacement for ribbon, but in order to use that, you need to be able to disable ribbon through the option stated earlier which doesn't seem to work for some reason.
The problem is that we can't really downgrade without losing the records and I don't really see a fix anywhere or I might just be overlooking something stupid.
EDIT: Excluding spring-cloud-starter-netflix-ribbon from spring-cloud-starter-consul-discovery fixed the issue for us!
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-consul-discovery</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
</exclusion>
</exclusions>
</dependency>
Can you try excluding ribbon dependency as shown below
implementation('org.springframework.cloud:spring-cloud-starter-consul-discovery') {
exclude group: 'org.springframework.cloud', module: 'spring-cloud-starter-netflix-ribbon'
}