How can I make an archive of all the youtube videos that I've used on my Hugo blog?

I have a Hugo website/blog with content written in markdown, and I've added embedded several youtube videos. I'd like to keep an offline archive of videos, in case any of the youtube videos are taken down. How can I make an archive of youtube videos I've linked on my blog?

Solution

Once you've installed youtube-dl (e.g. brew install youtube-dl), you can run this one-liner (build for MacOS but probably also work on Linux or WSL).

This works for Hugo-type youtube embedding, which uses this form

{{< youtube 1UdI_eoDPKQ >}}

Here's the one-liner:

find . -type f -name "*.md" -exec grep -r 'youtube' {} + | awk -F "{{" '{print $NF}' | sed 's/ >}}//g' | sed 's/< youtube//g' | xargs youtube-dl --id

which will download videos to the local directory.

Steps

find: get all .md (markdown) files in the children folders of $cwd (recursive)
grep: find all instances of "youtube" in those markdown files
awk: split the row on the curly brackets, and take the last element
sed, sed: remove everything except the youtube video id
youtube-dl: download the list of video ids to $cwd