Python / Selenium - how do i stay signed in after calling a second driver.get()?

I have this code to log into cbt nuggets and afterwards i want to go into my playlists and collect some URLs

from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.support import ui



capabilities = DesiredCapabilities.CHROME
capabilities["goog:loggingPrefs"] = {"performance": "ALL"}  # chromedriver 75+

options = webdriver.ChromeOptions()
options.add_argument(f"user-data-dir={userdata_path}") #Path to your chrome profile

# options.add_experimental_option("excludeSwitches", ['enable-automation'])
# options.add_experimental_option("excludeSwitches", ["ignore-certificate-errors", "safebrowsing-disable-download-protection", "safebrowsing-disable-auto-update", "disable-client-side-phishing-detection"])

driver = webdriver.Chrome(executable_path=webdriver_path, options=options)

driver.get("https://www.cbtnuggets.com/login")
logs = driver.get_log("performance")

def page_is_loaded(driver):
    return driver.find_element_by_tag_name("body") != None

#wait=ui.WebDriverWait(driver,300)
driver.implicitly_wait(10)
#wait.until(page_is_loaded)

USERNAME = driver.find_element_by_xpath('//*[@id="email"]')
USERNAME.send_keys("[email protected]")

PASSWORD = driver.find_element_by_xpath("/html/body/div[1]/div[2]/main/div/div[1]/form/fieldset/div[2]/input")
PASSWORD.send_keys("password")

Login_Button=driver.find_element_by_xpath("/html/body/div[1]/div[2]/main/div/div[1]/form/fieldset/button")
Login_Button.click()

driver.get("https://www.cbtnuggets.com/learn/it-training/playlist/nrn:playlist:user:5fcf88f463ebba00155acb18/2?autostart=1")

it all works as expected, but when the last driver.get() executes, i get thrown back to the login page, but when i manually enter the second URL in the address bar it works as expected without having to log in again. I dont know if this is a selenium issue, or if i am misunderstanding something about how HTTP Get works.

Solution

Have you tried to parse the login result? This might happen because the login request is not fully processed yet.

After Login_Button.click() you should check if the site is logged in successfully or not. You have many ways to check:

If the site redirects: check for the title of the page
If the site display a dialog: create a fluent wait to check for the dialog element to display
If you don't even bother to check, just add time.sleep(5). It's bad practice though.

After the check, now you use driver.get to go to the page you want.