I wanted to scrape data from a website which is "https://fhsaa.com/sports/2020/1/28/member_directory.aspx". The idea is to search school names using id or school name and then scrape the information. but when I used xpath or css_selecotr there is an exception "o such element: Unable to locate element: {"method":"xpath","selector":"//input[@name='school_name']"}
" I have used both xpath, css_selector and waits too but no success. Here is the code...
<fieldset class="border">
<legend class="border">School Filter</legend>
<form method="GET" action="https://fhsaahome.org/widget/school-directory/submit" accept-charset="UTF-8">
<div class="row">
<div class="col-md-4">
<select class="form-control" name="school_id"><option selected="selected" value="">--Select--</option><option value="714">Abundant Life Christian (Margate)</option><option value="665">Academy at the Lakes (Land O'Lakes)</option><option value="639">Academy Prep (St. Petersburg)</option><option value="4">Admiral Farragut (St. Petersburg)</option><option value="1191">AIE Charter (Miami Springs)</option>value="542">Zephyrhills</option><option value="1062">Zephyrhills Christian</option></select>
</div>
<div class="col-md-4">
<input class="form-control" placeholder="Enter School Name" name="school_name" type="text" value="">
</div>
<br><br>
<div class="col-md-2">
<input class="btn btn-secondary btn-md" type="submit" value="Search">
</div>
</div>
This is an iframe game, I see the website has called an API to continuously shuffle the xpath you are looking. I add a piece of code to switch between iframes and they worked:
iframe = driver.find_elements_by_tag_name('iframe')[0]
driver.switch_to.frame(iframe)
element = driver.find_element_by_xpath('//html/body/div[1]/fieldset/form/div/div[2]/input')
This is the full program:
import os
from selenium import webdriver
from time import sleep
from random import randint
chromedriver = "C:\Program Files\Python39\Scripts\chromedriver"
os.environ["webdriver.chrome.driver"] = chromedriver
driver = webdriver.Chrome(chromedriver)
url = 'https://fhsaa.com/sports/2020/1/28/member_directory.aspx'
driver.get(url)
sleep(randint(2,3))
iframe = driver.find_elements_by_tag_name('iframe')[0]
driver.switch_to.frame(iframe)
element = driver.find_element_by_xpath('//html/body/div[1]/fieldset/form/div/div[2]/input')
element.send_keys('yourschool')
sleep(randint(2,3))