Search code examples
python-3.xgoogle-chromeselenium-webdriverautomationgoogle-colaboratory

how to automate execution of colab notebook using python selenium

my code is:

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains

driver = webdriver.Chrome()

url = "https://colab.research.google.com/drive/any colab link=sharing"

driver.get(url)
driver.implicitly_wait(10)

first_run_botton = driver.find_elements(by=By.CSS_SELECTOR, value = r'div > div.cell-execution-indicator > iron-icon')[0]

print(first_run_botton)

first_run_botton.click()
driver.implicitly_wait(10)

error is Exception has occurred: IndexError
list index out of range
  File "C:\Users\Administrator\Desktop\python projects\python yt_video auto\automate colab.py", line 14, in <module>
    first_run_botton = driver.find_elements(by=By.CSS_SELECTOR, value = r'div > div.cell-execution-indicator > iron-icon')[0]
                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

all run botton have same css selector and xpath, css selector and xpath is copied from chrome please help!!

Solution

  • I think you are trying to click on the Run cell button for snippets, if you look at colab html you can cleary see that its part of a shadow DOM that why you are not able to access it directly in Selenium. if there is any element which comes under #shadow-root(open) we need to first find the parent of that and then get the shadowroot and then you can access the elements inside shadowroot

    enter image description here

    Below solution should work i have tested them on sample colab link below

    #1 If you are using Selenium > 4.1 , you can access shadowRoot thorugh sleenium itself as it has support for it from version > 4.1, This should click on Run button for first snippet , you can update locators as per your link

    driver.get("https://colab.research.google.com/notebooks/snippets/importing_libraries.ipynb")

driver.implicitly_wait(20)
# Find Shadow Element First
shadow = driver.find_element(By.XPATH, '(//colab-run-button[@aria-label="Run cell"])[2]').shadow_root

runButton = shadow.find_element(By.CSS_SELECTOR, 'div[class="cell-execution-indicator"] > iron-icon')
runButton.click()

    If you using selenium 3. version then we need use javascript to first get the shadowroot and then traverse through it to get the run button

    driver.get("https://colab.research.google.com/notebooks/snippets/importing_libraries.ipynb")

driver.implicitly_wait(20)

# Find Shadow Element First
shadow = driver.find_element(By.XPATH, '(//colab-run-button[@aria-label="Run cell"])[2]')

shadow = driver.execute_script('return arguments[0].shadowRoot;', shadow)

runButton = shadow.find_element(By.CSS_SELECTOR, 'div[class="cell-execution-indicator"] > iron-icon')
runButton.click()

    Read this to understand more about Shadow DOM