How do I record audio on multiple platforms with Uno Platform

Question

I am trying to record audio and process it afterwards. As I understand there is no unified API I could use to access microphone across different platforms. I am aiming at WASM, UWP and Android.

My approach is to record audio with platform-specific code.

For UWP I can use MediaCapture class as described here: link. I have implemented this part and it's working just fine.

For Android it should be straight forward as well. I can use Android.Media.MediaRecorder as shown here: link. I am not sure how to get the recorded audio file afterwards.

As for WASM I am completely lost here. I suppose I could use some javascript library to record the sound or vmsg library as described here link. But I have no idea how to get the recorded data into C# code for further processing.

Is there some material I should read to better understand the topic. Do I overlook some important details? Or is there an easy way how to record audio in xamarin.android and wasm?

UPDATE: I have successfully implemented audio capture on Android using Android.Media.MediaRecorder and I can simply access the file after I am finished recording.

Answer 1

I finally have the recording working on all three platforms (UWP, Android, WASM). The goal here is to obtain raw PCM audio data with a 48 kHz sample rate, 1 channel and 16 bits per sample.

UWP:

Global vars:

MediaCapture audioCapture = new MediaCapture();
IRandomAccessStream buffer = new InMemoryRandomAccessStream();

Recording audio:

public async Task RecordAudio(int recordingLength, uint sampleRate, uint channels, uint bitsPerSample)
{
    //initialization
    await audioCapture.InitializeAsync(new MediaCaptureInitializationSettings { StreamingCaptureMode = StreamingCaptureMode.Audio });

    //encoding setup
    MediaEncodingProfile profile = MediaEncodingProfile.CreateWav(AudioEncodingQuality.Auto);
    profile.Audio = AudioEncodingProperties.CreatePcm(sampleRate, channels, bitsPerSample);

    //start recording
    await audioCapture.StartRecordToStreamAsync(profile, buffer);

    await Task.Delay(recordingLength * 1000);

    //stop recording
    await audioCapture.StopRecordAsync();
}

Obtain PCM data as an array of bytes:

public async Task<byte[]> GetPCMData()
{
    byte[] data = new byte[buffer.Size];
    DataReader dataReader = new DataReader(buffer.GetInputStreamAt(0));

    await dataReader.LoadAsync((uint)buffer.Size);
    dataReader.ReadBytes(data);
    return data;
}

ANDROID: Remember to check microphone permission first. Global vars:

byte[] buffer;
AudioRecord recorder;
int bufferLimit;

Recording audio:

public async Task RecordAudio(int recordingLength, uint sampleRate, uint channels, uint bitsPerSample)
{
    //initialization
    // seconds * (number of bytes bytes per 1 sample) * sampleRate
    bufferLimit = recordingLength * bitsPerSample/8 * sampleRate;

    //encoding setup
    ChannelIn channels = Parameters.Channels == 1 ? ChannelIn.Mono : ChannelIn.Stereo;
    if (bitsPerSample != 16)
        throw new Exception("Unsupported bits per sample value");

    recorder = new AudioRecord(
                  AudioSource.Mic,
                  (int)sampleRate,
                  channels,
                  Android.Media.Encoding.Pcm16bit,
                  bufferLimit
               );

    //start recording
    buffer = new byte[bufferLimit];
    int totalBytesRead = 0;
    recorder.StartRecording();

    //record until specified amount of bytes was read
    while(totalBytesRead < bufferLimit)
    {
        int bytesRead = recorder.Read(buffer, 0, bufferLimit);
        if (bytesRead < 0)
        {
            throw new Exception(String.Format("Exception code: {0}", bytesRead));
        }
        else
        {
            totalBytesRead += bytesRead;
        }
    }
    
    //stop recording
    recorder.Stop();
    recorder.Dispose();    
}

Obtain PCM data as an array of bytes:

public async Task<byte[]> GetPCMData()
{
    return buffer;
}

WASM:

I used RecordRTC javascript library to obtain audio data:

function record(recordingLength, sampleRate, channels) {
    navigator.mediaDevices.getUserMedia({
        audio: true
    }).then(async function (stream) {
        
        //initialization
        let recorder = RecordRTC(stream, {
            type: 'audio',
            mimeType: 'audio/wav',
            recorderType: StereoAudioRecorder,
            sampleRate: sampleRate,
            desiredSampRate: sampleRate,
            numberOfAudioChannels: channels

        });
        
        //start recording
        recorder.startRecording();

        const sleep = m => new Promise(r => setTimeout(r, m));
        await sleep(recordingLength * 1000);

        recorder.stopRecording(function () {
            //blob of PCM audio data that can be further processed
            let blob = recorder.getBlob();
            }

        });
    });

}

In WASM I invoked C# code from javascript with blob as base64 string to process blob of PCM audio data further in C# (How to invoke C# code from javascript can be found in Uno Platform docs).

I am aware, this is not the cleanest solution, but it is a working one.