Following my other post, I am wondernig if it is possible to do some process like MFCC extraction on the decoded audio packets. The code I use decode audio and video from mpeg-2 file using ffmpeg. Process on video is done using opencv as this library permits to grab frames on by one. I need to process the corresponding audio samples in the same time.
Thanks.
I've created a C++ audio engine named "Crosstalk".
Although it's referred to as an "audio engine", It's really just a real-time C++ data (floating point) processing engine. Crosstalk allows you to create and route systems in design-time and real-time. Basically, the engine takes care of all the data routing and gives you a simple platform for creating components through which the data gets processed (E.g. your "Audio Feed" component connected in parallel with the "Video Feed" component). As long as your branches are of equal total buffer length, they will be perfectly synchronized.
It's very easy to use. Here's an example of how to configure a system to play an mp3 file (The components used here are provided with the engine):
XtSystem system;
XtMp3Decoder mp3Decoder;
XtAudioDevice audioDevice;
long md = system.addComponent(&mp3Decoder);
long ad = system.addComponent(&audioDevice);
system.connOutToIn(md,0,ad,0);
system.connOutToIn(md,1,ad,1);
mp3Decoder.loadFile("../05 Tchaikovski-Swan Lake-Scene.mp3");
mp3Decoder.play();
You can check out the API documentation and licensing details here: http://www.adaptaudio.com/Crosstalk
EDIT (01-12-2012):
Crosstalk has been replaced by an open-source project called "DSPatch". DSPatch is essentially an upgraded version of the routing engine behind Crosstalk that is no longer limited to only audio processing. DSPatch allows you to create and route almost any type of process chain imaginable, and free for personal AND proprietary use :)