Search code examples
pythonpandasdataframeglobpathlib

How to create a pandas dataframe from one file (with any file name) located in a specified folder?

What's the best way to create a pandas dataframe from one file with any file name located in a specified folder?

I have used pathlib and it's not quite working as the output dataframe is not giving me anything.

from pathlib import Path
import pandas as pd

pth = r'C:\Users\HP\Desktop\IBM\New folder'
fle = Path(pth).glob('*.tsv')

someDf = pd.DataFrame(fle)
someDf

Edit:

I also tried doing the below, but the output dataframe combines all columns into one column separated by a backward slash. How do I fix this?

from pathlib import Path
import pandas as pd

pth = r'C:\Users\HP\Desktop\IBM\New folder'
fle = Path(pth).glob('*.tsv')

dfs = []
for filename in fle:
    dfs.append(pd.read_csv(filename))

dfs1 = pd.concat(dfs)
dfs1.head()

enter image description here

The way I did this seems complicated. Is there an easier way to do this?

Solution

  • The solution I found for this is as below. I missed the sep parameter in pd.read_csv().

    from pathlib import Path
import pandas as pd

pth = r'C:\Users\HP\Desktop\IBM\New folder'
fle = Path(pth).glob('*.tsv')

dfs = []
for filename in fle:
    dfs.append(pd.read_csv(filename, sep='\t'))

dfs1 = pd.concat(dfs)
dfs1.head()