Hey there Python Experts, I have used Beautiful Soup and REquests to scrape data from static web for my project. But for Dynamic contents i am unable to do the same. I have installed selenium for the same. But when i execute the below code; The code goes to sleep mode after opening browser. I can see only '1Test' in my op window.
Please help :)
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Ie(executable_path='IEDriverServer_Win32_3.150.1/IEDriverServer.exe')
print('1test')
driver.get('http://www.google.com')
print('2test')
driver.close()
print('3test')
driver.quit()
you need to temporarily stop the execution of the program by using
driver.sleep(<number of seconds>)
or by importing the time module and writing
time.sleep(<number of seconds>)
this needs to be done after every step.
this needs to be done the different elements on a webpage need time to load up and hence if called before they are fully loaded, selenium will not be able to access them and will raise a error