Only first iteration is a valid webm file in mediarecorder dataavailable listener

So I am developing an audio recorder for the browser to transcribe text using Open AI whisper. However only the first iteration of MediaRecorder dataavailable is saved as a valid .webm file, then they are saved as binaries. I have tried to hardcode .webm as well, and logging the blob.type which is printed as audio/webm.

I am creating a new file every 15 seconds using the mediarecorder timeslice option: mediaRecorder.start(15000);

Can anyone put me in the right direction what might be going wrong here?

Current code:

Frontend code:

window.handleSuccess = function (stream) {
    const options = {mimeType: 'audio/webm;codecs=opus'};
    const recordedChunks = [];
    const mediaRecorder = new MediaRecorder(stream, options);

    mediaRecorder.addEventListener('dataavailable', function (e) {

        const reader = new FileReader();
        reader.readAsArrayBuffer(e.data);
        reader.onloadend = function () {
            const blob = new Blob([new Uint8Array(reader.result)], { type: 'audio/webm' });

            recordedChunks.push(blob);

            // post data to server
            const formData = new FormData();
            formData.append('audio', blob, 'filename.webm');
            formData.append('audio_mime', blob.type);
            formData.append('meeting_id', meetingId);
            fetch('/api/audio', {
                method: 'POST',
                body: formData,
                accept: 'application/json',
            }).then((response) => {
                console.log(response);
            }).catch((error) => {
                console.error(error);
            });

        };
    });

Laravel backend code

    public function uploadAudio(Request $request): JsonResponse
    {
        $mime = $request->input('audio_mime');
        $extensionMap = [
            'audio/webm' => 'webm',
        ];
        $extension = $extensionMap[$mime] ?? 'webm';
        $audioName = time() . '.' . $extension;

        $request->audio->move(storage_path('audio'), $audioName);

        Audio::create([
            'audio_file' => $audioName,
            'meeting_id' => $request->get('meeting_id'),
        ]);

        return response()->json([
            'message' => 'Audio uploaded successfully',
            'audio' => $audioName,
        ], 201);
    }

Solution

The file produced by the MediaRecorder is only required to be a valid file if you combine all the chunks. The individual chunks don't have to be valid files.

In case of WebM the first chunk appears to be a valid file because WebM supports appending additional audio. In other words, the first chunk is a valid file. The first and the second chunk combined are a valid file, too. But the second chunk on its own isn't a valid file.

If you want to keep using WebM you could probably manually parse the first chunk to extract the header. This header can then be appended to every subsequent chunk to turn it into a valid file.

If you don't mind the larger file size you could also record the audio as PCM with an additional library. It makes adding headers way easier.

This is for example how it would work with extendable-media-recorder:

const uploadWav = (blob) => {
  // your upload logic ...
  console.log(blob.size, blob.type);
};

const mediaRecorder = new MediaRecorder(mediaStream, {
  mimeType: 'audio/wav',
});

let header;

mediaRecorder.addEventListener('dataavailable', async ({ data }) => {
  if (header === undefined) {
    header = (await data.arrayBuffer()).slice(0, 44);

    uploadWav(data);
  } else {
    const content = await data.arrayBuffer();

    uploadWav(new Blob([header, content], { type: data.type }));
  }
});
mediaRecorder.start(15000);

Here is a link to a simple demo project: https://stackblitz.com/edit/js-h48mqc?file=index.js.