I am trying to parse a cloudflare website using selenium. I can find individual elements on the page, but I did not find how to get the entire code of the page.
options = webdriver.ChromeOptions()
options.add_argument('user-agent=')
options.add_argument('--disable-blink-features=AutomationControlled')
s = Service(executable_path='')
driver = webdriver.Chrome(service=s, options=options)
try:
driver.get('https://mangalib.me/manga-list')
time.sleep(10)
print(driver.find_element_by_xpath(''))
except Exception as ex:
print(ex)
finally:
driver.close()
driver.quit()
to get the entire source code you just do:
driver.get('https://mangalib.me/manga-list')
html = driver.page_source
then you can do whatever you want with it