I'm using the ZMQ pattern dealer/router.
In my project, a router is an agent manager and a dealer is an agent.
So I have many dealers and only one router.
Each dealer can send its own request to the router and wait for the reply.
The router listens to one port so it must be able to identify the routers.
I know that we can use zmq_setsockopt for the dealer to give its an unique id. And if the dealer sends a request to the router, the router will receive its unique id and its request.
Imagine that the router has already had 10 requests coming from 10 dealers. Now the router wants to send a reply to the fifth dealer. How to do that?
There's a slight modification to what you've said...
[...] if the dealer sends a request to the router, the router will receive its unique id and its request.
(emphasis added)
The reality is that if you don't explicitly set the identity, the dealer automatically selects its own identity. It will still send this identity to the ROUTER socket in the first frame of the message, and you can still use it to send a message back to that dealer, you just have to keep track of it when it comes in.
You can read a little more about how this works in the guide. I recommend you read the whole thing, at least from the beginning through chapter 5, but the linked section covers how the communication works with ROUTER sockets.
You don't specify a language, but there are many examples from different languages on how to work with this here.