Search code examples
python-3.xsaveexport-to-csv

Is there a way to save a file that is written by a python script, if the python script is killed before it is completed?


I have been running a web scraper script written in Python. I had to terminate the Python script because of an issue with my internet connection. At the time, the script has run for almost 2-3 hours. I used a for loop to write the data into a CSV file. I had used 'file.close()' to save the file once the for loop is over; but as I terminated the program early, my time of two hours have wasted.

Once I tried to delete the newly created CSV file(its size is 0kB), it is said 'The action can't be completed because the file is open in Python'. I thought that all the data I extracted is now on the RAM.(maybe that's why I don't get the permission to close the 0kB sized CSV file?)

So, is there any way to access those data and write the data into the above-mentioned CSV file? (Otherwise, I will have to run to the same program for another two hours and wait for the results)

Here's my code!

#! python3.8
fileCsv = open('newCsv.csv','w',newline='')
outputWriter = csv.writer(fileCsv)


for i in range(100,000): # whatever range
  num, name = 10000, 'hello' # The data extracted from the website
  ourputWriter.writerow([num,name])
  time.sleep(1)

fileCsv.close() # My program was terminated before this line, in the for loop




Solution

  • Using with should help here.

    with open('newCsv.csv','w') as wr:
        for i in range(100,000): # whatever range
          num, name = 10000, 'hello' # The data extracted from the website
          wr.writerow([num,name])
          time.sleep(1)