Search code examples
pythonxpathweb-scrapingsplinter

How to find an element's position using XPath?


I've been trying to write a simple script in order to upload 200+ links to a website I'm working in (I have poor knowledge on python and even poorer in HTML, of course I wasn't working as a web developer, I just need to upload these links).

Well, the situation I'm in is the following: I am using Splinter(therefore, Python) in order to navigate in the website. Certain section titles of this website will be compared with values I have in a .csv table.

For instance, in this screenshot, I am looking for this link /admin/pages/5, and I would like to compare the link's title (Explorar subpáginas de 'MA111 - Cálculo I') with my .CSV table. The problem is the link's title doesn't appear in the website.

To find the link I would guess that I should use find_by_xpath(), but I don't know how to do it. I would guess it's something like this link.

I would appreciate any help! I hope I have made myself clear.


Solution

  • You first need to define how are you detecting that url, so for example, "it is always to the right of certain button", or "it is the second row in a table", that way you can build the respective xpath (which is a path to follow inside the DOM.

    I am not entirely sure, but this could give you the solution

    url = browser.find_by_xpath('//td[@class="children"]/a')[0]['href']
    

    if you are finding a tag by the link name for example, try this:

    url = browser.find_by_xpath('//a[contains(@title, "MA111 - Cálculo I")]')[0]['href']
    

    If you check there, the xpath says "find in the entire DOM // a tag named a which contains "MA111 - Cálculo I" in the title attribute.