Search code examples
htmllinuxwgetmp4

using Wget to get every mp4 file in a website link, that checks every link in that link for embeded mp4 links

So I want to use wget to download all the mp4 files from this website:

https://coursevania.courses.workers.dev/[coursevania.com]Python%20for%20Data%20Science%20and%20Machine%20Learning%20Bootcamp/

However, when I try it always downloads the index file, and doesn't check any link in the website. It also takes about some seconds for the websites links to load. You can try it in the website. I tried 

wget -r -l2 -nd -A'*.mp4*' https://coursevania.courses.workers.dev/[coursevania.com]Python%20for%20Data%20Science%20and%20Machine%20Learning%20Bootcamp/

I honestly have no idea what I am doing, I've also tested for the robots.txt file and set it to ignore but it still returns the same result, downloads the index file and stops. I'm really new to wget.

Solution

  • It's not possible.

    Since wget is not a javascript engine, it will not be able to know the link to the mp4 resources (not until the javascript in the page has been executed). A view of the page source shows that it's entirely generated via this script.
    If you really need to do this recursively you should consider using javascript. Have a look at https://phantomjs.org/.