How to use SRQ when connected to more than one connections. lets say, there are three connections namely process 0, 1 and 2. for creating SRQ, we need to call
struct ibv_srq *ibv_create_srq(struct ibv_pd *pd, struct ibv_srq_init_attr *srq_init_attr);
for the above call we need to provide a protection domain, in my knowledge, protection domain is allocated specific to each connection by a call
ibv_alloc_pd(id->verbs)
in which id is created for each channel. basically, my question is how to assign a SRQ to different QP belonging to different Connection ID with different protection domain, or in other words, different connections can have a same protection domain?
A single SRQ or PD cannot be shared among multiple processes. The point of SRQ is to reduce the number of receives that need to be posted when a single process has many QPs. And sharing an SRQ among processes wouldn't really make sense. When you post a receive to the SRQ, the buffer would have to come from the address space of one of the processes. Then if a receive on a different process's QP used that work request, that process wouldn't be able to access the data it had received (since the buffer belongs to a different process).
You can of course share SRQs and all other verbs data structures between multiple threads in a single process.
The only verbs object that can be shared among multiple processes is an XRC domain (XRCD). See ibv_open_xrcd() etc.