Share HTML DOM audio element with THREE.AudioListener

Question

I have a WebGL experience that has a component that reacts to the audio that is playing. We also have standard HTML DOM elements that need to listen to the audio so we can render subtitling.

In THREE we can create a THREE.AudioListener like this:

// create an AudioListener and add it to the camera
const listener = new THREE.AudioListener();
camera.add( listener );

// create a global audio source
const sound = new THREE.Audio( listener );

// load a sound and set it as the Audio object's buffer
const audioLoader = new THREE.AudioLoader();
audioLoader.load( 'sounds/ambient.ogg', function( buffer ) {
    sound.setBuffer( buffer );
    sound.setLoop(true);
    sound.setVolume(0.5);
    sound.play();
});

Is there a way to get the audio source from the DOM instead of loading it with the THREE.AudioLoader? Or tell the THREE.AudioListener to listen for the audio from the DOM element instead?

Answer 1

The Three.js Audio object (and all its related classes) use the WebAudio API. I believe you can get the WebAudio context via the Audio.context property, which gives you access to any native WebAudio command.

Secondly, based on the "Loading Sound" section from this demo, it looks like you can select an <audio> tag and pass it to the WebAudio API via createMediaElementSource():

// create a global audio source
const listener = new THREE.AudioListener();
camera.add( listener );
const audio = new THREE.Audio( listener );

// Get WebAudio context
const audioContext = audio.context;

// get the <audio> element
const audioElement = document.querySelector('audio');

// pass it into the audio context
const track = audioContext.createMediaElementSource(audioElement);

Edit:

I just realized Three.js already does this for you without accessing the context manually via the Audio.setMediaElementSource method, so you could skip a step:

// create a global audio source
const listener = new THREE.AudioListener();
camera.add( listener );
const audio = new THREE.Audio( listener );

// get the <audio> element
const audioElement = document.querySelector('audio');
const track = audio.setMediaElementSource(audioElement);