Currently I am using spring cloud Netflix stack for development of my microservices based application. where I have used different services provided by Netflix stack like
EUREKA : for service registration and discovery
ZUUL : for proxy gateway and
RIBBON : for load balancing
now we want to use Apache kafka for inter-communication between microservices where kafka will have different topics to which our microservices will subscribe, now real problem starts here is how the load balancing, proxy gateway and eureka will work here.
few questions I have in mind which are really confusing me are
1. how ribbon load balancing will work here, since while using HTTP we are using @loadbalanced restTemplate but now this will not be the case.
2. how service calls will be distributed among different instances of a same service
3. whether Zuul and Eureka are needed here or we just need to ignore these and go with kafka load balancing and service discovery through topics.
I gooled but not found any satisfying answer if any expert here can help me will be appreciated, if there is any similar questions that can also help.
Thanks in advance.
For your above use-case , if you are going to use kafka for inter microservices communication , there is no need for any spring-cloud-netflix component. You can publish to a topic and have consumers in microservices consume from the topic. Load balancing will automatically happen depending upon number of partitions in the topic.
For example , lets consider your topic name is test and it has 4 partitions . If you have deployed 4 microservices with each consisting of a single Kafka Consumer consuming from topic test then each consumer will consume from 1 partition on the topic test. So , load balancing will automatically happen.
The spring-cloud-netflix components are mainly meant for inter-microservices communication when there is network calls involved between microservices .
For example - Lets consider two applications A and B . You have 4 instances of Application A deployed and 4 instances of Application B deployed . The entry-point for the consumers of your system is Application A . Here , you will use an Api Gateway like Zuul . As for Eureka , you will have all your instances deployed register in it. When a request comes into Zuul which needs to be forwarded to Application A . All the instances of Application A will be fetched from Eureka (4 in our case) and then being provided to a load balancer Ribbon who will chose which url should be called . Zuul will then forward the request to that instance of Application A.
Some links you should look at are-
https://github.com/Netflix/zuul/wiki/How-it-Works
https://github.com/Netflix/ribbon/wiki/Working-with-load-balancers