Failed to set the 'buffer' property on 'AudioBufferSourceNode': Failed to convert value to 'AudioBuffer'

Question

I am trying to use socket.io and mediastream to send audio in realtime between clients. I was able to record, send that data to the server and then to a different client. But when I try to convert the arraybuffers to audiobuffers and then connect them to an audio output device, i get the error message

Failed to set the 'buffer' property on 'AudioBufferSourceNode': Failed to convert value to 'AudioBuffer'.

I dont understand why this is happening. Here is some of my code.

client side

const socket = io();
const audioCtx = new AudioContext()
const audioC = new AudioContext()

    navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia

    if (navigator.getUserMedia) {
      console.log('getUserMedia supported.')

      navigator.getUserMedia(
        { audio: true },
        stream => {
          const source = audioCtx.createMediaStreamSource(stream)
          const processor = audioCtx.createScriptProcessor(2048, 1, 1)

          source.connect(processor)
          processor.connect(audioCtx.destination)

          processor.onaudioprocess = e => {
            socket.emit('Voice', e.inputBuffer.getChannelData(0))
          }

          socket.on('Voice', msg => {
            const source = audioC.createBufferSource();
            source.buffer = msg;
            source.connect(audioC.destination);
            source.start()
          })
          audioCtx.resume()
        },
        err => console.error(err)
      )
    }

server side

//Server shit
const path = require('path');
const http = require('http');
const express = require('express');
const socketio = require('socket.io');
const { stringify } = require('querystring');
const { compileFunction } = require('vm');
const app = express();
const server = http.createServer(app);
const io = socketio(server);
app.use(express.static(path.join(__dirname, 'public')));
const PORT = 3000 || process.env.PORT;
server.listen(PORT,  console.log('Server is running on port 3000'))

//variables
var clients = [{}]
var rooms = [{}]

io.on('connection', (socket) => {

    socket.on('Voice', (msg) => {
        console.log(msg)
        socket.emit('Voice', msg)
    })

})

at this point in the code..

socket.on('Voice', msg => {
            const source = audioC.createBufferSource();
            source.buffer = msg;
            source.connect(audioC.destination);
            source.start()
          })

when I log the msg, it gives me an arrayBuffer, and then i use the process above to convert the arrayBuffer to an audioBuffer yet it just gives me the same error message. Ive been banging my head against a wall for days now to try and figure this out so if anyone has a solution I would extremely appriciate it.

Answer 1

The data that you're sending is the channelData which is a Float32Array.

socket.emit('Voice', e.inputBuffer.getChannelData(0))

I'm not that familiar with socket.io but I assume the data arrives on the other end as a Float32Array, too. In that case you will have to build an AudioBuffer with that data before you can use it with an AudioBufferSourceNode.

const audioBuffer = new AudioBuffer({
    length: msg.length,
    sampleRate: sampleRateOfTheAudioContextOfTheSender
});

audioBuffer.copyToChannel(msg, 0, 0);

That audioBuffer can then be used with the AudioBufferSourceNode.

source.buffer = audioBuffer;