For a hobby project of me i'm looking to implement a group voice chat feature. Pretty straight forward: I'm running a server to which multiple clients (mobile) can connect to. Some Clients are in the same "group" and I want them to have an audio chat feature.
I already set up a UDP server client with C# to which clients can connect to. I have successfully implemented audio distribution between clients over the server and the rudimentary features are working pretty well.
I am not sure if i'm going in the right direction with this approach though. I'm stuck with the implementation of mixing the different voices for example (two people talking simultaneously and another is listening to both). I don't really know how I can mix both voices together and generate different outputs for different clients - above mentioned example: The two people talking should only receive the input of the other person while the one not talking should receive a mix of the other two talking.
What is the best server sided structure for this? Should i maybe go in a completely different direction and work with SIP? I'm having a hard time finding suitable resources for this problem online and i'm really stuck.
Thanks for your help!
Let me suggest using standards in your application. If (as I strongly recommend) your application is a web application, WebRTC make your job dramatically easier. Please see in WebRTC samples some ideas that I'm sure will inspire you, including multiple peer connections
If you are only interested in group calls, you could install a PBX server as asterisk that has a powerfull set of conference capabilities. You could use a SIP library on top of WebRTC in clients (e.g. sip.js, sipml5) to connect via SIP to asterisk and obtain conference services. This could sound daunting but the code for calling a conference room can be reduced to very few lines and asterisk can be easily installed in a linux box in a real machine, or in a virtual one or in a docker container.
If you prefer fat clients, I suggest using a SIP library as PJSIP (that incidentally is the base of the new SIP stack for asterisk). Proprietary solutions ride against the future while standard ones are pushed by it.