Search code examples
pythonseleniumweb-scrapingbotsnosuchelementexception

Can't get element link with Selenium in Python


I'm trying to click on the item but I can't seem to get the link from the element within this div. I feel like I've tried everything.

Here's an example of the HTML code:

<div class="tile css-yrcab6-Tile e1yt6rrx0" data-testid="product-tile">
  <a style="color:black" href="/product-name">...</a> #what I want to click
</div>

Here is a couple ways I've tried to click:

#1

item_div = item.find_element_by_tag_name('div')
item_link = item_div.find_element_by_tag_name('a')
#this returns NoElement for element with tag 'a'

#2

item_link = item.find_element_by_css_selector('div.tile a')
#again, returns no element

I want to click on every item on a page using a for loop

Here's the code so far:

def do_items(self):
    for page in range(self.maxPages):
        maincontent = driver.find_element_by_id('main-content')
        browsegrid = maincontent.find_element_by_class_name('browse-grid')
        items_on_page = browsegrid.find_elements_by_tag_name('div')
        for item in items_on_page:
            item_div = item.find_element_by_tag_name('div')
            item_link = item_div.find_element_by_tag_name('a')
            item_link.click()
            time.sleep(5)
            break
        break #breaks for the sake of testing, i only want to click the first item to see it works.

I'm pretty new to selenium so I don't entirely understand why it can't find the element. According to my code, I'm right on top of it..


Solution

  • //div[@data-testid='product-tile']/a
    

    A simple xpath to use to click.

    Now you either want to do this by grabbing all the a tags and using driver.get() or clicking each element. If it goes to another page you'll be using driver.switchto another window and then driver.close(). If it doesn't then you can just use driver.back() to repeatly grab the elements and then use it's index to click().