How to store URLS from text file in an array to visit with selenium browser?

Question

I have tried this:

def ad_open_file(ad_chrome):
    ad_url_list = []
    for line in ad_url_list:
        ad_url_list.append(line)

So I would like the array is looking like this:

ad_url_list = ['https://www.link.org', 'https://www.link.org']

After that I would like to visit every URL with the selenium browser with time.sleep(5) in between. Is is done by a for loop?

Can anyone help me out with this?

Answer 1

To visit every URL with the Selenium browser & sleep between visits, you can try this:

from selenium import webdriver
from time import sleep

# first get lines from the file -- assuming ad_chrome is your file path?
with open(ad_chrome) as f:

    # lines is a list containing each line in the file, as a list item
    lines = f.readlines()

    # start the webdriver
    driver=webdriver.Chrome()

    # now loop through lines and visit each URL
    for url in lines:

        # visit the URL
        driver.get(url.rstrip()) # call rstrip() to remove all trailing whitespace

        # wait 5 seconds
        sleep(5)

Hopefully this can get you started. We do not need to save the file contents to an array, because we can just iterate over each line in the file, so putting the file lines in an array is a bit redundant.

We call rstrip() on each line to remove the trailing whitespace and newline characters that may be present in the file.

This code makes the assumption that your file is something like:

www.someurl.com
www.anotherurl.com
www.google.com

etc..