ARCore - playing YouTube video over augmented image surface

Question

I'm trying to play a video from YouTube over an augmented image target surface in an android app built with Google ARCore library. With a local video file I have no problem, I use the MediaPlayer object as below:

MediaPlayer mediaPlayer = MediaPlayer.create(context, R.raw.my_vid_local_file);
mediaPlayer.setSurface(texture.getSurface());
mediaPlayer.setLooping(true);
if (!mediaPlayer.isPlaying()) {
  mediaPlayer.start();

  // Wait to set the renderable until the first frame of the  video becomes available.
  // This prevents the renderable from briefly appearing as a black quad before the video
  // plays.
  texture.getSurfaceTexture().setOnFrameAvailableListener(
        (SurfaceTexture surfaceTexture) -> {
          videoPlayerTexture.getNow(null).getMaterial().setExternalTexture("videoTexture", texture);
          videoPlayerTexture.getNow(null).getMaterial().setFloat4("keyColor", CHROMA_KEY_COLOR);
          mediaPlayerNode.setRenderable(videoPlayerTexture.getNow(null));
          texture.getSurfaceTexture().setOnFrameAvailableListener(null);
        });
} else {
  mediaPlayerNode.setRenderable(videoPlayerTexture.getNow(null));
}

All works fine, but I would love get the video data source from YouTube so I can change it without download again my app from store.

I read this thread: Android Java Youtube API v3.0 get video RTSP link about taking an RTSP link within YouTube API v3 but when I put it as my MediaPlayer object datasource I get an Exception.

I modified the code as below:

MediaPlayer mediaPlayer = new MediaPlayer();
try {
  mediaPlayer.setDataSource(context, Uri.parse("https://r8---sn-nx5cvox-hpay.googlevideo.com/videoplayback?expire=...and so on"));
  mediaPlayer.setOnPreparedListener(new MediaPlayer.OnPreparedListener() {
    @Override
    public void onPrepared(MediaPlayer mp) {
      if (!mediaPlayer.isPlaying()) {
        mediaPlayer.start();

        // Wait to set the renderable until the first frame of the  video becomes available.
        // This prevents the renderable from briefly appearing as a black quad before the video
        // plays.
        texture.getSurfaceTexture().setOnFrameAvailableListener(
                (SurfaceTexture surfaceTexture) -> {
                  videoPlayerTexture.getNow(null).getMaterial().setExternalTexture("videoTexture", texture);
                  videoPlayerTexture.getNow(null).getMaterial().setFloat4("keyColor", CHROMA_KEY_COLOR);
                  mediaPlayerNode.setRenderable(videoPlayerTexture.getNow(null));
                  texture.getSurfaceTexture().setOnFrameAvailableListener(null);
                });
      } else {
        mediaPlayerNode.setRenderable(videoPlayerTexture.getNow(null));
      }
    }
  });
  mediaPlayer.prepare();
}catch(Exception e) {
  e.printStackTrace();
  Log.e(TAG, e.getMessage());
}

And the error is:

D/MediaHTTPConnectionEx: [response code] STATUS CODE:403
I/native: tracking.cc:3027 no history. confidence : 1.0
D/MediaHTTPConnectionEx: [ResponseMessage]:Forbidden
E/MediaHTTPConnectionEx: MediaHTTPConnectionEx
com.lge.media.MediaHTTPConnectionEx 411 seekTo exception toString: java.io.IOException
D/MediaHTTPConnectionEx: connect
D/MediaHTTPConnectionEx: disconnecting
D/MediaHTTPConnectionEx: disconnected
D/MediaHTTPConnectionEx: [seekToEx] offset:0/mCurrentOffset:-1
D/MediaHTTPConnectionEx: proxy null port 0
I/native: tracking.cc:3027 no history. confidence : 1.0
D/MediaHTTPConnectionEx: [response code] STATUS CODE:403
D/MediaHTTPConnectionEx: [ResponseMessage]:Forbidden
E/MediaHTTPConnectionEx: MediaHTTPConnectionEx com.lge.media.MediaHTTPConnectionEx 411 seekTo exception toString: java.io.IOException
D/MediaHTTPConnectionEx: connect
D/MediaHTTPConnectionEx: disconnecting
D/MediaHTTPConnectionEx: disconnected
...

There would be a way to use YouTube API KEY or am I completely wrong approach?

Thank you in advice

Carlo Tassi

Answer 1

YouTube is designed to be played either using its own webpage, on another webpage via embedded HTML5 or using the YouTube mobile API's.

The key common factor is that they all retain the YouTube branding and look and feel.

There are some examples of ways to get the raw video URL so you can play it directly (e.g. https://stackoverflow.com/a/9740907/334402) but if you use the raw video URL and MediaPlayer, you lose the Youtube look and feel and generally this is not something they support.

If you just want to ability to play a video that you have some other source for or that you can host yourself, then your approach looks good.