Search code examples
pythonseleniumweb-scrapingyoutube

noob I am following this python selenium webscraping tutorial /watch?v=lTypMlVBFM4 I don't know why it won't get the title from youtube and print it


I am trying to scrap this site https://www.youtube.com/@JohnWatsonRooney/videos for the video titles.

This short explination does not help with the problem of the code but I thought I would say it anyway. I am doing this so I can get my foot in the door to then scrap other sites for prices on stuff so I can make a program that gives me a link to the cheapest priced product from a list several sites as well as a ranking of all the other products and there names incase that wasn't the one I was looking for.

here is the terminal output.

Traceback (most recent call last):
File "C:\\Users\\PRO\\Documents\\dynamic_web_scraping_test.py", line 11, in \<module\>
videos = driver.find_element("name", "style-scope ytd-rich-grid-media")
File "C:\\Users\\PRO\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python310\\site-packages\\selenium\\webdriver\\remote\\webdriver.py", line 830, in find_element
return self.execute(Command.FIND_ELEMENT, {"using": by, "value": value})\["value"\]
File "C:\\Users\\PRO\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python310\\site-packages\\selenium\\webdriver\\remote\\webdriver.py", line 440, in execute
self.error_handler.check_response(response)
File "C:\\Users\\PRO\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python310\\site-packages\\selenium\\webdriver\\remote\\errorhandler.py", line 245, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: \[name="style-scope ytd-rich-grid-media"\]
Stacktrace:
RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8
WebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:180:5
NoSuchElementError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:392:5
element.find/\</\<@chrome://remote/content/marionette/element.sys.mjs:275:16

C:\\Users\\PRO\\Documents\>

here is my code

import time
from selenium import webdriver
url1 = 'https://www.youtube.com/@JohnWatsonRooney/videos'
url2 = 'https://www.youtube.com/@MyCraftyDen/videos'
driver = webdriver.Firefox()
driver.get(url1)
time.sleep(10)
videos = driver.find_element("name", "style-scope ytd-rich-grid-media")
for video in videos:
titles = video.find_element_by_xpath('//\*\[@id="video-title-link"\]').text
print(title)

I don't know what to try I did my best looking up this problem before posting. I know waiting for the browser to load the page doesnt work hense the time.sleep(10) in my code.


Solution

  • If you are looking to find the links of all the videos in the given page, you can use below code

    import time
    from selenium import webdriver
    from selenium.webdriver.common.by import By
    
    url1 = 'https://www.youtube.com/@JohnWatsonRooney/videos'
    url2 = 'https://www.youtube.com/@MyCraftyDen/videos'
    driver = webdriver.Firefox()
    driver.get(url1)
    time.sleep(10)
    videos = driver.find_elements(By.ID, "video-title-link")
    for video in videos:
        title = video.get_attribute("href")
        print(title)