"This page can not be loaded" Selenium button redirect link not loading

Question

I am taking an url as an input and enter some text in search bar than clicking first search-suggestion and clicking on search icon using selenium in Python.

But now the selenium driver is not able to load the redirected url.(This page can not be loaded error ). It belongs to same domain. When I am opening same link myself in other window. It is working.

Environment:

OS => Mac arm-64,
Driver =>ChromeDriver Stable
Version: 129.0.6668.58 (r1343869)

def extractSocietyData(societyName:str)->None:
    
    driver = webdriver.Chrome()
    
    driver.get("https://www.99acres.com/")
    
    time.sleep(5)
    
    print(driver.title)
    
    driver.execute_script("window.scrollBy(0, 1000);") 
    
    time.sleep(2)
    
    search_bar = driver.find_element(By.ID,"keyword")
    print(search_bar.get_attribute('placeholder'))
    
    # Click search bar to enter data
    search_bar.click()
    
    # Enter Society Name in search
    search_bar.send_keys(societyName)
    
    # Wait for suggestions
    time.sleep(10)
    
    # Click first available suggestion
    search_suggestions =driver.find_element(By.CSS_SELECTOR,"#suggestions_custom")
    first_search_suggestion = search_suggestions.find_element(By.TAG_NAME, 'li')
    first_search_suggestion.click()

    # Press enter search_bar
    search_bar.send_keys(Keys.RETURN)

    time.sleep(10)
    # Error occurs here
    # This page can not be loaded
    url = driver.current_url
    
    extract_data(driver.page_source)

I was expecting the redirected page will be loaded but it didn't instead of it I got this error "This page can not be loaded."

Answer 1

I think the server is blocking the Selenium driver bot. So the page was not getting loaded whenever I tried.

I found a walkaround to load the page. After being redirected, I closed the driver, instantiated a new Chrome driver, and used that to load the driver. Although it is not the best approach, this is working.

url = driver.current_url

driver.close()
time.sleep(5)
driver = webdriver.Chrome()
driver.get(url)