Search code examples
pythonfileiostorefile-format

python read all files from a folder and write the file name and other info into a txt file

I have 30911 html files. I need to do webscraping and then save the info into a txt file named index.txt. It should look like

filename1, title, t1, date, p1
filename2, title, t1, date, p1
filename3, title, t1, date, p2
and so on...

I only want filename, but output gave me path+filename.

Solution

  • Your problem is that filename is filepath in reality, in order to get the filename you could use os module

    os.path.basename('filepath')

    so in order to write to the file:

    indexFile.write(os.path.basename(filename)+ ', ' + title.get_text(strip=True) + ', '+ ticker.get_text(strip=True) + ', ' + d_date.get_text(strip=True) + ', ' + parti_names + '\n')