My understanding of microservices is that you can "just" spin-up multiple instances of the same service, as required.
My question is, then, if you have x instances of a microservice, spread over y servers, how does anything calling an API on the microservice know where to find it?
One approach I've seen is to have some kind of discovery service (fixed IP address that can route to an instance, like a load-balancer); but surely that just pushes the problem back a layer - the discovery service then needs to know where everything is / when it fails, etc.? And what about high-availability for the discovery service (if you have multiple instances of that, you're back to not knowing where anything is again)
Another approach might be to use pub/sub messaging, but again you still need to know where the queue manager is (with high-availability, etc.); so you've still essentially got the same problem - and responses to queries are trickier with that approach.
Another related issue is, if you have a microservice that is pulling from a STOMP feed, how would you make that HA? you can't just have x instances of that service, or you'll be subscribing and reading the data x times, which means you end up duplicating the data when it's passed on to downstream systems. So you would want some kind of active/passive approach to that, right? Which means you need something to manage that failover, which again gives a single point of failure?
You can have a service registry where a service registers itself on startup. The registry may listen for broadcasts by other services for the registration events so the registry's location does not need to be fixed somewhere in the service.
All other Services can then use this registry to look up the available instances by querying the registry. The registry may also act as a DNS resolver so your services can use DNS to resolve other services by name and also automatically load balance between multiple hosts.
Another way may be to reverse the responibility and the service wanting to contact the other service broadcasts requests that are answered with the needed information by the other service (which basically is what DNS does).
One readily available solution to this is Consul by Hashi Corp. Have a look at the features it provides which may be additionally useful, e.g. healthchecks.