How to stream audio file with opentok?

Question

In opentok, with OT.initPublisher, you only can pass a deviceId to the audioSource. Does someone know a method to stream an audio file ?

For example, I have done this:

navigator.getUserMedia({audio: true, video: false},
function(stream) {
    var context = new AudioContext();
    var microphone = context.createMediaStreamSource(stream);
    var backgroundMusic = context.createMediaElementSource(document.getElementById("song"));
    var mixedOutput = context.createMediaStreamDestination();
    microphone.connect(mixedOutput);
    backgroundMusic.connect(mixedOutput);
},
handleError);

Like this, I can have a stream with the voice and my music but how to put this stream to a publisher ? Is it possible or is there another way to do this ?

Answer 1

Update: There is now an official way to do this, using the videoSource and audioSource properties provided to OT.initPublisher, please see the documentation: https://tokbox.com/developer/sdks/js/reference/OT.html#initPublisher

This is an example of how to stream a canvas element as a video track: https://github.com/opentok/opentok-web-samples/tree/master/Publish-Canvas

You can apply the same technique to stream an audio track.

---

Old Answer:

It's not currently possible with the officially supported API but there is a way to do it.

Please see the TokBox blog post about Camera Filters: https://tokbox.com/blog/camera-filters-in-opentok-for-web/

In order to modify the stream before it reaches the OpenTok JS SDK we use the mockGetUserMedia function to intercept the stream:

https://github.com/aullman/opentok-camera-filters/blob/master/src/mock-get-user-media.js

You could invoke mockGetUserMedia with a function which does your audio mixing. Something like this:

mockGetUserMedia(function(originalStream) {
  var context = new AudioContext();
  var microphone = context.createMediaStreamSource(originalStream);
  var backgroundMusic = context.createMediaElementSource(document.getElementById("song"));
  var mixedOutput = context.createMediaStreamDestination();
  microphone.connect(mixedOutput);
  backgroundMusic.connect(mixedOutput);

  var stream = mixedOutput.stream;
  originalStream.getVideoTracks().map(function(track) {
    stream.addTrack(track);
  });
  return stream;
});

Note: I have not tested this function but it should lead you in the right direction. Remember that this technique is error prone and not officially supported by TokBox.

We are currently working on a new feature which will enable this use case but I cannot give a time estimate of when it will be available.