Search code examples
pythonbeautifulsoupexport-to-csv

Csv doesn't generate when it


It doesn't generate.

when it did, the file was blank.

    import csv
import requests 
from bs4 import BeautifulSoup as bs
for i in range(3):
        r = requests.get('https://www.nairaland.com/search/afonja/0/0/0/{}'.format(i))
        soup = bs(r.content, 'lxml')
        occurrences = len(soup.select('.highlight')) #if soup.select_one('.highlight') else 
    z=[occurrences]
    with open('z.csv', 'w') as csv_file:
        writer = csv.writer(csv_file)
        writer.writerow([z]) 
   

Update: someone has solved it. It was clear to that user.


Solution

  • You need to collect all the occurrence values into a list, not overwrite the list each time through the loop. After the loop, write the list to the CSV file.

    And since z is already a list, you don't need to write [z] when writing it to the file.

    import csv
    import requests 
    from bs4 import BeautifulSoup as bs
    z = []
    for i in range(3):
        r = requests.get('https://www.nairaland.com/search/afonja/0/0/0/{}'.format(i))
        soup = bs(r.content, 'lxml')
        occurrences = len(soup.select('.highlight'))
        z.append(occurrences)
    with open('z.csv', 'w') as csv_file:
        writer = csv.writer(csv_file)
        writer.writerow(z)