Safari behaving very weirdy with HTML5 audio tag

Question

I am building a site where I have a HTML5 audio element where I use javascript to dynamically update the source depending on what track the users click. This is wrapped by the web audio API which does an fft and uses that to control a scene in three js.

The site works correctly in chrome and firefox, however in safari the audio is extremely problematic.

The first issue was safari won't accept source as a child node of audio if your updating it dynamically, you must put src in the audio tag. I'm checking for safari/IOS in my js and either setting a opus or mp3 audio source depending on the result. I tried using m4a as the file format but it wouldn't play, despite the browser supporting m4a?

The persisting issues are:

The .pause() method does not work on the audio element, it simply plays from the beginning of the track again when the user resumes play.

Trying to set the .currentTime to allow the user to skip the track in does nothing in safari. However it is reading currentTime as its setting the position of a play head.

When the animate() function (which is called recursively with requestAnimationFrame) is running it is extremely slow before it begins playing anything, over a minute delay, same on ios but not as severe and ios doesn't suffer the other issues.

The poly count on the scene is approx 250k which is quite high, however it would be weird if the polycount on the canvas was making audio slow to play, notwithstanding the other issues with the audio whilst the requestAnimationFrame is not running.

Excuse the sloppy code I didn't have much time to write this:

const playTrack = (trackNumber) => {
    audio.src = `assets/audio/${trackNumber}.${fileType}`;
    audio.load();
    audio.play();
};

audio.ontimeupdate = () => {
    const left = audio.currentTime * segment;
    playHead.style.width = `${left}px`;
};

playbar.addEventListener("click", (e) => {
    const pos = e.x - container.left;
    audio.currentTime =  pos / segment;
}, false);

    tracks.forEach(track => {
    track.addEventListener("click", (e) => {
        e.preventDefault();

        //pause or load
        if (!audio.paused && nowPlaying === track) {
            //pause track
            audio.pause();
            track.children[0].classList.remove('fa-pause');
            track.children[0].classList.add('fa-play');
        } else if (audio.paused && nowPlaying === track) {
            //play track
            audio.play();
            track.children[0].classList.remove('fa-play');
            track.children[0].classList.add('fa-pause');
        } else {
            //load and play track
            instructions.style.display = "none";
            trackNumber = parseInt(track.getAttribute('data-value'), 10);
            playTrack(trackNumber);
            nowPlaying = track;

            iconFlick(track);
        }

        trackDuration = duration[trackNumber];
        segment = width / trackDuration;

        if (clickFirst === 0) {
            clickFirst = 10;
            setupNodes(audio)
        }

        trackGlobal = track;
    });

const setupNodes = (audio) => {
    const audioCtx = new (window.AudioContext || window.webkitAudioContext)();
    const source = audioCtx.createMediaElementSource(audio);

    analyser = audioCtx.createAnalyser();
    source.connect(analyser);

    analyser.fftSize = 256;
    bufferLength = analyser.frequencyBinCount;
    dataArray = new Uint8Array(bufferLength);

    audioBegin = 1;

    analyser.connect(audioCtx.destination);
};

please let me know if you spot anything, this has had me stumped.

Answer 1

You might have better luck and more control if you don't use the <audio> tag. You'll need to make your own play button and slider if you want those controls. And use Web Audio API by itself to download and decode and play a track.

var url = "whatever.mp3";
var source = audioCtx.createBufferSource();
var buffers = {};

var request = new XMLHttpRequest();
request.open('GET', url, true);
request.responseType = 'arraybuffer';

request.onload = function () {
    audioCtx.decodeAudioData(request.response, function (buffer) {
        source.buffer = buffer;
        source.connect(audioCtx.destination);
        source.start(audioCtx.currentTime);

        // save this for later, so you can replay it without downloading
        buffers[url] = buffer; 

    }, function () {
        console.warn("error loading sound url: " + url);
    });
}
request.send();