So I have a web app using Google's Shaka player for video playback. It uses the MediaSource API to stream audio and video segments.
I want to do some audio processing on the raw PCM data of my audio tracks. I see that it fetches audio segments in .mp4 format, then appends these in the form of an ArrayBuffer to a SourceBuffer, which is then appended to the MediaSource and played. Is there some way to hijack this flow, decode the data to PCM, do some processing, then pipe my processed PCM back in to the SourceBuffer? I see there's the AudioContext.decodeAudioData API for getting segments from .mp4 to PCM, but this returns an AudioBufferSourceNode and AudioBuffer, which can't be appended to my regular SourceBuffer. Is there some way to get around this?
I figure I could re-encode my PCM data back to .mp4, but that seems like a huge waste.
In Chrome 64 you can define your own AudioWorkletProcessor, and use it for AudioWorkletNode instantiation.
See AudioWorklet Examples for further reference.