Search code examples
pythonselenium-webdriverfindelementmailinator

Python Selenium cannot get anything when using .text method


HTML/CSS I'm trying to extract, the highlighted text is what I'm trying to extract

I'm specifically trying to get a verification code out of an email from Mailinator, however, I have tried getting and printing text from anything and nothing appears to be storing when I use the .text function. Additionally, I can only use find_elements(By.XPATH, "example) and not find_elements_by_xpath("example") as an example (same for css_selector, etc). For reference, I'm using the most current version of Python and Visual Studio Code as my environtment. Does anyone know what may be the issue? I've tried a number of sources, but I can't find a similar case to what I'm getting.

I should add that I've also tried scraping and I'm getting the same result, although that was more of a half try of just trying to find all elements that would fit a regex and store them with a loop.

I'm not getting an error message, it just doesn't appear to be storing text, or, at least, it isn't printing it. It prints a \n. It isn't just this code, by the way. I've tried several websites, including Google's homepage, and I can't seem to store and print text from anywhere on the web. For the record, this is the code with the credentials removed (if you see any other errors, please let me know):

from selenium import webdriver
from selenium.webdriver.common.by import By
import time
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
**# from faker import Faker
**# import re****

options = webdriver.ChromeOptions()
options.add_experimental_option('excludeSwitches', ['enable-logging'])
driver = webdriver.Chrome(options=options)
wait = WebDriverWait(driver, 20)
username = '*'
driver.get("https://www.mailinator.com/v4/login.jsp")
wait.until(EC.element_to_be_clickable((By.XPATH, "/html/body/main/div/div/div/section/form/div[2]/input"))).send_keys(
    "*")
wait.until(EC.element_to_be_clickable((By.XPATH, "/html/body/main/div/div/div/section/form/div[3]/input"))).send_keys(
    "*")
wait.until(EC.element_to_be_clickable((By.XPATH, "/html/body/main/div/div/div/section/form/div[4]/a[1]"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//*[contains(@id,'row_"+ username +"')]/td[2]"))).click()
iframe = driver.find_element('tag name', 'iframe')
print('line21 executed')
driver.switch_to.frame(iframe)
print('line23 executed')
email_text = driver.find_element('tag name', 'body').text
print('line25 executed')
print(email_text)
print('line27 executed')

I'm trying to get text stored so I can extract a verification code to put in as part of a registration process while performing the BDD for a project, but I've hit this stonewall for several hours at this point and cannot resolve the issue.

EDIT:

I've updated the code to the following after suggestions and I will show the output at the bottom:

   from selenium import webdriver
    from selenium.webdriver.common.by import By
    import time
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC
    # from faker import Faker
    # import re

    options = webdriver.ChromeOptions()
    options.add_experimental_option('excludeSwitches', ['enable-logging'])
    driver = webdriver.Chrome(options=options)
    wait = WebDriverWait(driver, 20)
    username = '*'
    driver.get("https://www.mailinator.com/v4/login.jsp")
    wait.until(EC.element_to_be_clickable((By.XPATH, "/html/body/main/div/div/div/section/form/div[2]/input"))).send_keys(
        "*")
    wait.until(EC.element_to_be_clickable((By.XPATH, "/html/body/main/div/div/div/section/form/div[3]/input"))).send_keys(
        "*")
    wait.until(EC.element_to_be_clickable((By.XPATH, "/html/body/main/div/div/div/section/form/div[4]/a[1]"))).click()
    wait.until(EC.element_to_be_clickable((By.XPATH, "//*[contains(@id,'row_"+ username +"')]/td[2]"))).click()
    iframe = driver.find_element(By.ID, 'html_msg_body')
    print('line21 executed')
    driver.switch_to.frame(iframe)
    print('line23 executed')
    email_text = wait.until(EC.visibility_of_element_located(('tag name', 'body'))).text
    print('line25 executed')
    print(email_text)
    print('line27 executed')

Output: line21 executed line23 executed line25 executed

line27 executed

EDIT 2:

I've made the code to a public Mailinator. This is the following code to the email:

from selenium import webdriver
from selenium.webdriver.common.by import By
import time
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# from faker import Faker
# import re

options = webdriver.ChromeOptions()
options.add_experimental_option('excludeSwitches', ['enable-logging'])
driver = webdriver.Chrome(options=options)
wait = WebDriverWait(driver, 20)
driver.get("https://www.mailinator.com/v4/public/inboxes.jsp?to=testingfor042223")
wait.until(EC.element_to_be_clickable((By.XPATH, '//*[@id="row_testingfor042223-1682214679-12524595"]/td[2]'))).click()
# wait.until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[1]/main/div[2]/div[3]/div/div[4]/div/div/table/tbody/tr/td[2]"))).click()
iframe = driver.find_element(By.ID, 'html_msg_body')
print('line21 executed')
driver.switch_to.frame(iframe)
print('line23 executed')
body = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "body")))
print('line25 executed')
print(body.text)
print('line27 executed')

Solution

  • With access to the email itself, I was able to get it working. The problem was that the page loads and then loads again, causing the original BODY to become stale. So, I grabbed the original BODY tag, waited for it to become stale, grabbed it again, and printed the text. I've verified the code below is working.

    NOTE: I left in but commented out the lines that log in if you want to use that later, after resetting the URL.

    from selenium import webdriver
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support import expected_conditions as EC
    from selenium.webdriver.support.wait import WebDriverWait
    
    driver = webdriver.Chrome()
    driver.maximize_window()
    driver.get("https://www.mailinator.com/v4/public/inboxes.jsp?to=testingfor042223")
    
    username = '*'
    wait = WebDriverWait(driver, 10)
    # wait.until(EC.visibility_of_element_located((By.ID, "many_login_email"))).send_keys("email")
    # wait.until(EC.visibility_of_element_located((By.ID, "many_login_password"))).send_keys("password")
    # wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[aria-label='Login link']"))).click()
    wait.until(EC.element_to_be_clickable((By.XPATH, "//td[contains(text(),'Your verification code')]"))).click()
    
    wait.until(EC.frame_to_be_available_and_switch_to_it((By.ID, "html_msg_body")))
    body = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "body")))
    wait.until(EC.staleness_of(body))
    body = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "body")))
    print(body.text)