Search code examples
pythonseleniumbotsrankingbrowser-automation

Retriving value from href using python selenium

<a class="yt-simple-endpoint style-scope yt-formatted-string" spellcheck="false" href="/channel/UC8butISFwT-Wl7EV0hUK0BQ" dir="auto">freeCodeCamp.org</a>

I am new to selenium and trying to make a youtube rank check bot! I am trying to get the href value from here so that I can compare it with the channel name and print out the proper ranking number but the outputs that I am getting are not correct. The outputs I am getting are 2,5 where I should be getting 6,7.

Can anyone please tell me where/what am I doing wrong? And what can be done to solve this problem? Advance thanks

Screenshot attached below to see the rankings enter image description here

from selenium import webdriver
import time
channel_name = 'freeCodeCamp.org' #channel name
driver = webdriver.Chrome(r"C:\\Users\\user\\PycharmProjects\\YoutubeRankCheckBot\\Drivers\\chromedriver.exe")
driver.get("http://youtube.com")
driver.maximize_window()

search_bar = driver.find_element_by_id("search")
search_bar.send_keys("React JS") #Inserting text input in a automation way
search_button = driver.find_element_by_id("search-icon-legacy")
search_button.click()

time.sleep(5)


video_list = driver.find_elements_by_xpath('//a[contains(@href,"/channel/UC8butISFwT-Wl7EV0hUK0BQ")]')
print(video_list)

for index, channel in enumerate(video_list):
    if channel.text  == channel_name:
        print(index)
Solution 
  •  video_list = driver.find_elements_by_xpath('//a[@class="style-scope ytd-video-renderer"]')
video_url = [video_list.get_attribute('href').replace('https://www.youtube.com', '') for video_list in video_list]
print(video_url)

for index, channel in enumerate(video_url):
    if channel == channel_id:
        print(index)

    Here channel_id is nothing but /channel/UC8butISFwT-Wl7EV0hUK0BQ for the channel name freeCodeCamp.org.

    1. First line gives all the element of a tag
    2. From a list of step 1, find the href element. It will give you the complete URL for e.g. https://www.youtube.com/channel/UC8butISFwT-Wl7EV0hUK0BQ. so replacing https://www.youtube.com and extracting into video_url
    3. Enumerate with video_url and print the index