Search code examples
pythonseleniumcloudflare

How to get page html code using selenium?


I am trying to parse a cloudflare website using selenium. I can find individual elements on the page, but I did not find how to get the entire code of the page.

options = webdriver.ChromeOptions()
options.add_argument('user-agent=')
options.add_argument('--disable-blink-features=AutomationControlled')

s = Service(executable_path='')
driver = webdriver.Chrome(service=s, options=options)

try:
    driver.get('https://mangalib.me/manga-list')
    time.sleep(10)
    print(driver.find_element_by_xpath(''))
except Exception as ex:
    print(ex)
finally:
    driver.close()
    driver.quit()

Solution

  • to get the entire source code you just do:

    driver.get('https://mangalib.me/manga-list')
    html = driver.page_source
    

    then you can do whatever you want with it