Search code examples
pythonpandascsvweb-scrapingreddit

Is there a way to save a csv as a new csv without overwriting the old one?


I am scraping a subreddit's comments and would like to save them in a csv file using pandas. I am able to save comments in all posts into one csv file. But, I would like to save each post's comments in its own csv. Using a for loop, it simply overwrites the previous csv.

Is there a function to save the csv with a new name or just not overwrite? I am using pandas in python. Below is my method.

  #Retrieve Comments
  def comments(self,posts,reddit):

    '''Method scrapes all first level comments, then all second level comments, etc'''

    for id in posts.id:
      all_comments = []
      all_comments.clear() #just to be sure
      sub = reddit.submission(id=id)
      sub.comments.replace_more(limit=0)

      for comment in sub.comments.list():
        all_comments.append(comment.body)

      all_comments = pd.DataFrame(all_comments,columns=['Comments'])

      #*****Path to Store CSV*****
      all_comments.to_csv(r'C:\Users\osama\Desktop\.csv')

    return

Solution

  • for index, id in enumerate(posts.id):
        # ...
        all_comments.to_csv(rf'C:\Users\osama\Desktop\{index}.csv')
    

    It will save the files as 0.csv, 1.csv, 2.csv and so on.

    Or:

    for post in posts:
        id = post.id
        # ...
        all_comments.to_csv(rf'C:\Users\osama\Desktop\{post.title}.csv')
    

    I think the second option should work with praw (which I can see that you're using). It should name the files after post titles.