I am scraping a subreddit's comments and would like to save them in a csv file using pandas. I am able to save comments in all posts into one csv file. But, I would like to save each post's comments in its own csv. Using a for loop, it simply overwrites the previous csv.
Is there a function to save the csv with a new name or just not overwrite? I am using pandas in python. Below is my method.
#Retrieve Comments
def comments(self,posts,reddit):
'''Method scrapes all first level comments, then all second level comments, etc'''
for id in posts.id:
all_comments = []
all_comments.clear() #just to be sure
sub = reddit.submission(id=id)
sub.comments.replace_more(limit=0)
for comment in sub.comments.list():
all_comments.append(comment.body)
all_comments = pd.DataFrame(all_comments,columns=['Comments'])
#*****Path to Store CSV*****
all_comments.to_csv(r'C:\Users\osama\Desktop\.csv')
return
for index, id in enumerate(posts.id):
# ...
all_comments.to_csv(rf'C:\Users\osama\Desktop\{index}.csv')
It will save the files as 0.csv, 1.csv, 2.csv and so on.
Or:
for post in posts:
id = post.id
# ...
all_comments.to_csv(rf'C:\Users\osama\Desktop\{post.title}.csv')
I think the second option should work with praw (which I can see that you're using). It should name the files after post titles.