Search code examples
iosobjective-cobjective-c++pitchsuperpowered

How to do real-time pitch shifting from mic with Superpowered?


I'm trying to make a pitch shift in real time from a microphone using superpowerd. I looked at the example that is for the file. Also tried to adapt it. I managed to change the sound, but it turned out very distorted with interference. What am I doing wrong? where to find more information on superpowered and timeStretching?

    static bool audioProcessing(void *clientdata,
                            float **buffers,
                            unsigned int inputChannels,
                            unsigned int outputChannels,
                            unsigned int numberOfSamples,
                            unsigned int samplerate,
                            uint64_t hostTime) {
    __unsafe_unretained Superpowered *self = (__bridge Superpowered *)clientdata;

    float tempBuffer[numberOfSamples * 2 + 16];
    SuperpoweredInterleave(buffers[0], buffers[1], tempBuffer, numberOfSamples);
    float *outputBuffer = tempBuffer;

    SuperpoweredAudiobufferlistElement inputBuffer;

    inputBuffer.samplePosition = 0;

    inputBuffer.startSample = 0;
    inputBuffer.samplesUsed = 0;
    inputBuffer.endSample = self->timeStretcher->numberOfInputSamplesNeeded;
    inputBuffer.buffers[0] = SuperpoweredAudiobufferPool::getBuffer(self->timeStretcher->numberOfInputSamplesNeeded * 8 + 64);
    inputBuffer.buffers[1] = inputBuffer.buffers[2] = inputBuffer.buffers[3] = NULL;

    memcpy((float *)inputBuffer.buffers[0], outputBuffer, numberOfSamples * 2 + 16);

    self->timeStretcher->process(&inputBuffer, self->outputBuffers);

    // Do we have some output?
    if (self->outputBuffers->makeSlice(0, self->outputBuffers->sampleLength)) {
        while (true) {  // Iterate on every output slice.
            // Get pointer to the output samples.
            int sampleCount = 0;
            float *timeStretchedAudio = (float *)self->outputBuffers->nextSliceItem(&sampleCount);
            if (!timeStretchedAudio) break;
            SuperpoweredDeInterleave(timeStretchedAudio, buffers[0], buffers[1], numberOfSamples);
        };

        // Clear the output buffer list.
        self->outputBuffers->clear();
    };

    return true;
}

Solution

  • I did the following:

    static bool audioProcessing(void *clientdata,
                                    float **buffers,
                                    unsigned int inputChannels,
                                    unsigned int outputChannels,
                                    unsigned int numberOfSamples,
                                    unsigned int samplerate,
                                    uint64_t hostTime) {
        __unsafe_unretained Superpowered *self = (__bridge Superpowered *)clientdata;
    
        SuperpoweredAudiobufferlistElement inputBuffer;
        inputBuffer.startSample = 0;
        inputBuffer.samplesUsed = 0;
        inputBuffer.endSample = numberOfSamples;
        inputBuffer.buffers[0] = SuperpoweredAudiobufferPool::getBuffer((unsigned int) (numberOfSamples * 8 + 64));
        inputBuffer.buffers[1] = inputBuffer.buffers[2] = inputBuffer.buffers[3] = NULL;
    
        SuperpoweredInterleave(buffers[0], buffers[1], (float *)inputBuffer.buffers[0], numberOfSamples);
    
        self->timeStretcher->process(&inputBuffer, self->outputBuffers);
    
        // Do we have some output?
        if (self->outputBuffers->makeSlice(0, self->outputBuffers->sampleLength)) {
            while (true) { // Iterate on every output slice.
                // Get pointer to the output samples.
                int numSamples = 0;
                float *timeStretchedAudio = (float *)self->outputBuffers->nextSliceItem(&numSamples);
                if (!timeStretchedAudio || *timeStretchedAudio == 0) {
                    break;
                }
    
                SuperpoweredDeInterleave(timeStretchedAudio, buffers[0], buffers[1], numSamples);
            }
            // Clear the output buffer list.
            self->outputBuffers->clear();
        }
    
        return true;
    }
    

    This might not work correctly when changing the speed also, but I wanted live pitch shifting only. People should be able to speak slower or faster themselves.