Playing a video with captions in Jupyter notebook

Question

How to play a video with captions in Jupyter notebook?

With code snippets from these post, I've tried to play a video inside jupyter notebook:

• How can I play a local video in my IPython notebook?
• how play mp4 video in google colab

from IPython.display import HTML

# Show video
compressed_path = 'team-rocket.video-compressed.mp4'
mp4 = open(compressed_path,'rb').read()
data_url = "data:video/mp4;base64," + b64encode(mp4).decode()


HTML("""
<video width=400 controls>
      <source src="%s" type="video/mp4">
</video>
""" % data_url)

[out]:

When I tried to add a .vtt file as caption, the option appears

from IPython.display import HTML

# Show video
compressed_path = 'team-rocket.video-compressed.mp4'
mp4 = open(compressed_path,'rb').read()
data_url = "data:video/mp4;base64," + b64encode(mp4).decode()


HTML("""
<video width=400 controls>
      <source src="%s" type="video/mp4">
      <track src="team-rocket.vtt" label="English" kind="captions" srclang="en" default >
</video>
""" % data_url)

[out]:

But the subtitles/captions are not present when playing the video. How to play a video with captions in Jupyter notebook?

The files used in the examples above are on:

• team-rocket.vtt
• team-rocket.video-compressed.mp4 file can be found on https://colab.research.google.com/drive/1IHgo9HWRc8tGqjmwWGunzxXDVhPaGay_?usp=sharing

Answer 1

Try this:

from IPython.display import HTML
from base64 import b64encode

video_path = 'team-rocket.video-compressed.mp4'
captions_path = 'team-rocket.vtt'

with open(video_path, 'rb') as f:
    video_data = f.read()
    video_base64 = b64encode(video_data).decode()

with open(captions_path, 'r') as f:
    captions_data = f.read()
    captions_base64 = b64encode(captions_data.encode('utf-8')).decode()

video_html = f"""
<video width="640" height="360" controls>
    <source src="data:video/mp4;base64,{video_base64}" type="video/mp4">
    <track src="data:text/vtt;base64,{captions_base64}" kind="captions" srclang="en" label="English" default>
</video>
"""

HTML(video_html)

For some reason, explicitly streaming in the video + captions/subtitle while specifying the encoding would bypass the security issues that @igrinis' answer pointed out.

• "data:video/mp4;base64,{video_base64}"
• "data:text/vtt;base64,{captions_base64}"