I've built a demo of a voice-assistant that takes microphone data, passes it to an analyzer, then uses .getByteFrequencyData()
to show visuals. It works as follows:
Live version here: https://dyadstudios.com/playground/daysi/
The way I've achieved this is as follows:
var audioContext = (window.AudioContext) ? new AudioContext() : new window["webkitAudioContext"]();
var analyser = audioContext.createAnalyser();
analyser.fftSize = Math.pow(2, 9); // 512
var sourceMic = undefined; // Microphone stream source
var sourceMp3 = undefined; // MP3 buffer source
// Browser requests mic access
window.navigator.mediaDevices.getUserMedia({audio: true}).then((stream) => {
sourceMic = audioContext.createMediaStreamSource(stream)
})
// 1. Mic button pressed, start listening
listen() {
audioContext.resume();
// Connect mic to analyser
if (sourceMic) {
sourceMic.connect(analyser);
}
}
// 2. Disconnect mic, play mp3
answer(mp3AudioBuffer) {
if (sourceMic) {
// Disconnect mic to prevent audio feedback
sourceMic.disconnect();
}
// Play mp3
sourceMp3 = audioContext.createBufferSource();
sourceMp3.onended = mp3StreamEnded;
sourceMp3.buffer = mp3AudioBuffer;
sourceMp3.connect(analyser);
sourceMp3.start(0);
// Connect to speakers to hear MP3
analyser.connect(audioContext.destination);
}
// 3. MP3 has ended
mp3StreamEnded() {
sourceMp3.disconnect();
// Disconnect speakers (prevents mic feedback)
analyser.disconnect();
}
It works perfectly well on Firefox and Chrome, but OSX Safari 12.1 only gets microphone data the first time I press the button. Whenever I press the mic button on a second pass, the analyzer no longer gets microphone data, but MP3 data still works. It seems like connecting, disconnecting, and re-connecting the mic's AudioNode to the analyzer breaks it somehow. I checked and Safari supports AudioNode.connect()
as well as AudioNode.disconnect()
. I know Safari's WebAudio implementation is a bit outdated, is there a workaround to fix this issue?
There is indeed a bug in Safari which causes it to drop the signal if a MediaStreamAudioSourceNode is disconnected for some time. You can avoid this by just not disconnecting it as long as you might need it again. You can use a GainNode instead to mute the signal.
You could do this by introducing a new variable to control the volume.
const sourceMicVolume = audioContext.createGain();
sourceMicVolume.gain.value = 0;
Then you need to connect everything right away when you instantiate the sourceMic
.
sourceMic = audioContext.createMediaStreamSource(stream);
sourceMic.connect(sourceMicVolume);
sourceMicVolume.connect(analyser);
Inside your event handlers you would then only set the volume of the gain instead of (dis)connecting the nodes. Inside the listen()
function that would look like this:
if (sourceMic) {
sourceMicVolume.gain.value = 1;
}
And inside the answer()
function it would look like this:
if (sourceMic) {
sourceMicVolume.gain.value = 0;
}