I am trying to parse a website using Selenium/Capybara. Right now it looks like this:
session = Capybara::Session.new(:selenium)
session.visit "https://somesite.com/page1"
element = session.all( :css, '.table .row a' ).each do |el|
el.click
# get some element's data
session.evaluate_script('window.history.back()')
end
# repeat
Problem is, when I'm trying to get the data from the second page I am on, Capybara tells me that Either the element is no longer attached to the DOM or the page has been refreshed.
which absolutely makes sense, however I'm struggling to find a way to create a new DOM variable and parse it in the documentation.
Same thing happens if I navigate back trying to repeat the actions and click on the second link in a row. I assume I need to re-create the session or is there a better way?
To work like you're trying you're going to need to keep a counter and find the elements each time through your loop - something along the lines of
counter = 0
while (el = session.all( :css, '.table .row a', minimum: 1 )[counter]) do
el.click
# get some element's data
counter += 1
session.go_back
end
or if the links are just standard you could gather the hrefs and then just visit them
element = session.all( :css, '.table .row a', minimum:1 ).map {|a| a['href']} do |url|
session.visit(url)
# get some element's data
end