Search code examples
openglaudiogpgpuglblendfunc

Mixing Audio Using OpenGL

I want to mix two (or more) 16bit audio streams using OpenGL and I need a bit of help

Basically what I want to do is to put the audio data into texture which I draw to a frame buffer object and then read back. This is not a problem, however drawing the data in way that gives correct results is a bit more problematic.

I have basically two questions.

  1. In order to mix the data by drawing i need to use blending (alpha = 0.5), however the result should not have any alpha channel. So if I render to e.g. a frame buffer with the format RGB will alpha blending still work as I expect and the resulting alpha will not be written to the fbo? (I want to avoid having to read back the fbo for each render pass)

texture |sR|sG|sB|

framebuffer(before) |dR|dG|dB|

framebuffer(after) |dR*0.5+sR*0.5|dG*0.5+sG*0.5|dB*0.5+sB*0.5|

  1. The audio samples are signed 16bit integer values. Is it possible do signed calculations this way? Or will I need to first convert the values to unsigned on the cpu, draw them, and then make them signed again on the cpu?

EDIT:

I was a bit unclear. My hardware is restricted to OpenGL 3.3 hardware. I would prefer to not use CUDA or OpenCL, since I'm alrdy using OpenGL for other stuff.

Each audio sample will be rendered in seperate passes, which means that it has to "mix" with whats already been rendered to the frame buffer. The problem is how the output from the pixel shader is written to the framebuffer (this blending is not accessible through programmable shaders, as far as i know, and one has to use glBlendFunc).

EDIT2:

Each audio sample will be rendered in different passes, so only one audio sample will be available in the shader at a time, which means that they need to be accumulated in the FBO.

foreach(var audio_sample in audio_samples)
     draw(audio_sample);

and not

for(int n = 0; n < audio_samples.size(); ++n)
{
      glActiveTexture(GL_TEXTURE0 + n);
      glBindTexture(audio_sample);
}
draw_everything();
Solution

    1. You should be able to do blending even if the destination buffer does not have alpha. That said, rendering to non-power-of-two sizes (rgb16 = 6bytes/pixel) usually incurs performance penalties.

    2. Signed is not your typical render target format, but it does exist in the OpenGL 4.0 specification (Table 3.12, called RGB16_SNORM or RGB16I, depending on whether you want a normalized representation or not).

    As a side note, you also have glBlendFunc(GL_CONSTANT_ALPHA,GL_ONE_MINUS_CONSTANT_ALPHA) to not even have to specify an alpha per-pixel. That may not be available on all GL implementations though.