Search code examples
pythoncsvbeautifulsoupattributeerror

No attribute replace with csv file strings

I have a csv file of many url's. I am writing a program to extract information from all of these sites.

Right now I open the csv file, read it, and append every link to a list.

links = []
with open("x.csv", 'r') as file:
    urls = csv.reader(file, delimiter=',')
    for url in urls:
        links.append(url)

I then iterate through this list and am trying to use the links as a string and extract information from each link. 

for link in links:
    url = link
    response = requests.get(url)

The problem is I kept getting an error saying: requests.exceptions.InvalidSchema: No connection adapters were found for "['https://link']" each time I tried to run the code and I figured it was because the string contained brackets ([ ]) and quotes ('), so I tried to replace these characters and get each string to be just the url information. 

for link in links:
    link.replace("'", "")
    link.replace("[", "")
    link.replace("]", "")
    url = link
    response = requests.get(url)

However when I did that I got an error that said: AttributeError: 'list' object has no attribute 'replace'

Any ideas on what the issue may be or a better way to iterate through a list of url's?

Solution

  • Just use ''.join(link) to return list as a string.

    for link in links:
    url =''.join(link)
    response = requests.get(url)