Trying to gather a list, using Python, of the Amazon Prime available movies from website:
I have tried to locate the title of the movie but returning 0. This must be from the page loading JavaScript first, I've looked in NETWORK but can't work out what I'm looking for.
I've attempted the following code:
from requests import get
url = 'https://www.amazon.co.uk/s?i=instant-video&bbn=3010085031&rh=n%3A3010085031%2Cp_85%3A3282143031%2Cp_72%3A3289786031&dc&adult-product=0&field-genre=-family&field-review_count=3-&field-ways_to_watch=7448662031&p_n_entity_type=9739952031&qid=1557596014&qs-av_request_type=4&qs-is-prime-customer=0&rnid=3289782031&ref=sr_nr_p_72_4'
response = get(url)
from bs4 import BeautifulSoup
html_soup = BeautifulSoup(response.text, 'lxml')
type(html_soup)
movie_containers = html_soup.find_all('span', class_ = 'a-size-medium a-color-base a-text-normal')
print(type(movie_containers))
print(len(movie_containers))
I've then tried to loop around:
for n in soup.find_all('span', {'class':'a-size-medium a-color-base a-text-normal'}):
title.append (n.text)
Any help would be great. Thank you.
Results should extract title and link for each movie.
You must include a header of the get request.
import requests
from bs4 import BeautifulSoup
header = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36"}
a = requests.get("https://www.amazon.co.uk/s?i=instant-video&bbn=3010085031&rh=n%3A3010085031%2Cp_85%3A3282143031%2Cp_72%3A3289786031&dc&adult-product=0&field-genre=-family&field-review_count=3-&field-ways_to_watch=7448662031&p_n_entity_type=9739952031&qid=1557596014&qs-av_request_type=4&qs-is-prime-customer=0&rnid=3289782031&ref=sr_nr_p_72_4", headers=header)
b = BeautifulSoup(a.text, "html.parser")
for c in b.find_all("span", class_="a-size-medium"):
print(c.text)