Search code examples
pythonhtmlweb-scrapingget

How to download video using Python requests module?

I am trying to download video shown at bottom of this site using python requests module. I can find the video URL. However, when I am trying to use it outside the page it is giving 404 status code.

Can someone help how to scrape video from the site ?

Thanks in advance

Solution

  • To download the mp4 videos with requests, set Referer HTTP header:

    import requests
from bs4 import BeautifulSoup

url = "https://drswamyplabvideo.com"
headers = {"Referer": "https://drswamyplabvideo.com/"}

soup = BeautifulSoup(requests.get(url).content, "html.parser")
for v in soup.select("video source[src]"):
    print("Downloading {}".format(v["src"]))
    with open(v["src"].split("/")[-1].strip(), "wb") as f_out:
        f_out.write(requests.get(v["src"].strip(), headers=headers).content)

    Prints:

    Downloading https://drswamyplabvideo.com/videos/Demo video.mp4
Downloading https://drswamyplabvideo.com/videos/CNS Day 1 by Nosheen Part 1.mp4
Downloading https://drswamyplabvideo.com/videos/CNS Part 5.mp4 
Downloading https://drswamyplabvideo.com/videos/Lady with fracture wrist Talk to son.mp4

    and saves the videos.