Extracting audio from a video file

Question

Edit: This post is no duplicate of mine. I am trying to extract the audio data as binary, got no problems with playing the audio file as separate as I mentioned before.

I am trying to extract audio from a video file on client-side by using Web Audio Api.

var audioContext = new(window.AudioContext || window.webkitAudioContext)();
fileData = new Blob([input.files[0]]);
var videoFileAsBuffer = new Promise(getBuffer);
videoFileAsBuffer.then(function (data) {
    audioContext.decodeAudioData(data).then(function (decodedAudioData) {
        mySoundBuffer = decodedAudioData;
        soundSource = audioContext.createBufferSource();
        soundSource.buffer = mySoundBuffer;
        // soundSource.connect(audioContext.destination);
        // soundSource.start();
    });

When I uncomment the two lines at the end, I am hearing the sound of the uploaded video file. Though, when I create a link to download the file with the help of getChannelData method, it's almost the same size as video file.

I was expecting decodedAudioData to have only audio binary data, and send that to my webservice, which is the only one I need. However, that didn't work out as I expected. Anyone knows a way to extract audio of a video file on client-side? Thanks in advance.

Here is the getBuffer method in case anyone wants to know:

function getBuffer(resolve) {
    var reader = new FileReader();
    reader.onload = function () {
        var arrayBuffer = reader.result;
        resolve(arrayBuffer);
    }
    reader.readAsArrayBuffer(fileData);
}

Edit: I was able to decode the video file and get audiobuffer by using OfflineAudioContext inside decodeAudioData function.

var offlineAudioContext = new OfflineAudioContext(2, 44100 * 100, 44100);
var soundSource = offlineAudioContext.createBufferSource();
...
soundSource.connect(offlineAudioContext.destination);
soundSource.start();
offlineAudioContext.startRendering().then(function (renderedBuffer) {
    console.log(renderedBuffer); // outputs audiobuffer
    var song = audioContext.createBufferSource();
    song.buffer = renderedBuffer;
    song.connect(audioContext.destination);
    song.start();
}).catch(function (err) {
    console.log('Rendering failed: ' + err);
});

renderedBuffer is an audiobuffer, had no problem outputting the data, tested with Audacity's import raw data option. But the problem is, the new file (filled with renderedBuffer.getChannelData(0)) has higher size than the original video has. Isn't that supposed to have lower size since it only contains audio of the video file?

Answer 1

Ok, I actually had the answer already. Raw audio data is huge, that's why it's even greater than the video itself in size.

var offlineAudioContext = new OfflineAudioContext(numberOfChannels, sampleRate * duration, sampleRate);
var soundSource = offlineAudioContext.createBufferSource();
...
reader.readAsArrayBuffer(blob); // video file
reader.onload = function () {
  var videoFileAsBuffer = reader.result; // arraybuffer
  audioContext.decodeAudioData(videoFileAsBuffer).then(function (decodedAudioData) {
    myBuffer = decodedAudioData;
    soundSource.buffer = myBuffer;
    soundSource.connect(offlineAudioContext.destination);
    soundSource.start();

    offlineAudioContext.startRendering().then(function (renderedBuffer) {
      console.log(renderedBuffer); // outputs audiobuffer
    }).catch(function (err) {
      console.log('Rendering failed: ' + err);
    });
  });
};

After that, I was able to convert the audiobuffer (renderedbuffer) into a wav file by using audiobuffer-to-wav library. OfflineAudioContext is just needed to modify the cropped audio.

Edit: Here is the js fiddle example. decodedAudioData method will suffice if you don't want to override the audio data.