Search code examples
c++audiopcmpan

How to pan audio sample data naturally?

I'm developing Flutter plugin which is targeting only Android for now. It's kind of synthesis thing; Users can load audio file into memory, and they can adjust pitch (not pitch shift) and play multiple sound with the least delay using audio library called Oboe.

I managed to get PCM data from audio files which MediaCodec class supports, and also succeeded to handle pitch by manipulating playback via accessing PCM array manually too.

This PCM array is stored as float array, ranging from -1.0 to 1.0. I now want to support panning feature, just like what internal Android class such as SoundPool. I'm planning to follow how SoundPool is handling panning. There are 2 values I have to pass to SoundPool when performing panning effect : left, and right. These 2 values are float, and must range from 0.0 to 1.0.

For example, if I pass (1.0F, 0.0F), then users can hear sound only by left ear. (1.0F, 1.0F) will be normal (center). Panning wasn't problem... until I encountered handling stereo sounds. I know what to do to perform panning with stereo PCM data, but I don't know how to perform natural panning.

If I try to shift all sound to left side, then right channel of sound must be played in left side. In opposite, if I try to shift all sound to right side, then left channel of sound must be played in right side. I also noticed that there is thing called Panning Rule, which means that sound must be a little bit louder when it's shifted to side (about +3dB). I tried to find a way to perform natural panning effect, but I really couldn't find algorithm or reference of it.

Below is structure of float stereo PCM array, I actually didn't modify array when decoding audio files, so it should be common structure

[left_channel_sample_0, right_channel_sample_0, left_channel_sample_1, right_channel_sample_1,
...,
left_channel_sample_n, right_channel_sample_n]

and I have to pass this PCM array to audio stream like c++ code below

void PlayerQueue::renderStereo(float * audioData, int32_t numFrames) {
    for(int i = 0; i < numFrames; i++) {
        //When audio file is stereo...
        if(player->isStereo) {
            if((offset + i) * 2 + 1 < player->data.size()) {
                audioData[i * 2] += player->data.at((offset + i) * 2);
                audioData[i * 2 + 1] += player->data.at((offset + i) * 2 + 1);
            } else {
                //PCM data reached end
                break;
            }
        } else {
            //When audio file is mono...
            if(offset + i < player->data.size()) {
                audioData[i * 2] += player->data.at(offset + i);
                audioData[i * 2 + 1] += player->data.at(offset + i);
            } else {
                //PCM data reached end
                break;
            }
        }

        //Prevent overflow
        if(audioData[i * 2] > 1.0)
            audioData[i * 2] = 1.0;
        else if(audioData[i * 2] < -1.0)
            audioData[i * 2] = -1.0;

        if(audioData[i * 2 + 1] > 1.0)
            audioData[i * 2 + 1] = 1.0;
        else if(audioData[i * 2 + 1] < -1.0)
            audioData[i * 2 + 1] = -1.0;
    }

    //Add numFrames to offset, so it can continue playing PCM data in next session
    offset += numFrames;

    if(offset >= player->data.size()) {
        offset = 0;
        queueEnded = true;
    }
}

I excluded calculation of playback manipulating to simplify code. As you can see, I have to manually pass PCM data to audioData float array. I'm adding PCM data to perform mixing multiple sounds including same sound too.

  1. How to perform panning effect with this PCM array? It will be good if we can follow mechanisms of SoundPool, but it will be fine as long as I can perform panning effect properly. (EX: pan value can be just -1.0 to 1.0, 0 will mean centered)

  2. When applying Panning Rule, what is relationship between PCM and decibel? I know how to make sound louder, but I don't know how to make sound louder with exact decibel. Are there any formula for this?

Solution

  • Pan rules or pan laws are implemented a bit different from manufacturer to manufacturer.

    One implementation that is frequently used is that when sounds are panned fully to one side, that side is played at full volume, where as the other side is attenuated fully. if the sound is played at center, both sides are attenuated by roughly 3 decibels.

    to do this you can multiply the sound source by the calculated amplitude. e.g. (untested pseudo code)

    player->data.at((offset + i) * 2) * 1.0; // left signal at full volume
player->data.at((offset + i) * 2 + 1) * 0.0; // right signal fully attenuated

    To get the desired amplitudes you can use the sin function for the left channel and the cos function for the right channel.

    enter image description here

    notice that when the input to sin and cos is pi/4, that the amplitude is 0.707 on both sides. This will give you your attenuation on both sides of around 3 decibels.

    So all that is left to do is to map the range [-1, 1] to the range [0, pi/2] e.g. assuming you have a value for pan which is in the range [-1, 1]. (untested pseudo code)

    pan_mapped = ((pan + 1) / 2.0) * (Math.pi / 2.0);

left_amplitude = sin(pan_mapped);
right_amplitude = cos(pan_mapped);

    UPDATE:

    Another option frequently used (e.g. ProTools DAW) is to have a pan setting on each side. effectively treating the stereo source as 2 mono sources. This allows you to place the left source freely in the stereo field without affecting the right source.

    To do this you would: (untested pseudo code)

    left_output  += left_source(i)  * sin(left_pan)
right_output += left_source(i)  * cos(left_pan)
left_output  += right_source(i) * sin(right_pan)
right_output += right_source(i) * cos(right_pan)

    The setting of these 2 pans are are up to the operator and depend on the recording and desired effect. How you want to map this to a single pan control is up to you. I would just advise that when the pan is 0 (centred) that the left channel is played only on the left side and the right channel is only played on the right side. Else you would interfere with the original stereo recording.

    One possibility would be that the segment [-1, 0) controls the right pan, leaving the left side untouched. and vice versa for [0, 1].

    hPi = math.pi / 2.0
  
def stereoPan(x):
    if (x < 0.0):
        print("left source:")
        print(1.0) # amplitude to left channel
        print(0.0) # amplitude to right channel
        print("right source:")
        print(math.sin(abs(x) * hPi)) # amplitude to left channel
        print(math.cos(abs(x) * hPi)) # amplitude to right channel

    else:
        print("left source:")
        print(math.cos(x * hPi)) # amplitude to left channel
        print(math.sin(x * hPi)) # amplitude to right channel  
        print("right source:")
        print(0.0) # amplitude to left channel
        print(1.0) # amplitude to right channel