How to mix audio input devices in Qt

I'm new to Qt's multimedia library and in my application I want to mix audio from multiple input devices (e.g. microphone), in order to stream it via TCP.

As far as I know I have to obtain the specific QAudioDeviceInfo for all needed devices first - together with an according QAudioFormat object - and use this with QAudioInput. Then I simply call start() for every created QAudioInput object and read out pending bytes with readLine().

But how can I mix audio data of multiple devices to one buffer?

Solution

I am not sure if there is any Qt specific method / class to do this. However it's pretty simple to do it yourself.

The most basic way (assuming you are using PCM), you can simply add the two streams/buffers together word by word (if I recall they are 16-bit PCM words).

So if you have two input buffers:

int16 buff1[10];
int16 buff2[10];
int16 mixBuff[10];

// Fill them...
//... code goes here to read from the buffers ....

// Add them (effectively mix them)

for (int i = 0; i < 10; i++)
{
   mixBuff[i] = buff1[i] + buff2[i];
}

Now, this is very crude and does not take any scaling into consideration. So imagine buff1 and buff2 both use 80% of the dynamic range (call this full volume, beyond which you get distortion), then when you add them together you will get number overrun (i.e. 16-bit max is 65535 so 50000 + 50000 will be a over run).

Each time you mix you effectively need half the two inputs (so 65535 / 2 + 65535 / 2 = 65535... i.e. when you add them up you can't overrun). So your mix code is like this:

for (int i = 0; i < 10; i++)
{
   mixBuff[i] = (buff1[i] >> 1) + (buff2[i] >> 1);
}

There is much more you can do (noise removal etc...) but then the maths start getting a bit hairy. This is very simple. You can use the shift afterwards to increase / decrease volume as simple volume control if you want.

EDIT

One thing to note... you are using readline() (which the docs say reads the data out as ASCII). I always use read() which it does not state the "format" it is read out, but I am assuming binary. So this code may not work if you use readline() but I have never tried it. It works well for read(), you don't really want to be working in ASCII if you want to manipulate the data.