Search code examples
pythonoperating-systemtypeerrordata-cleaning

Error when trying to create a list of files


I have a folder that contains 20 csv files. Each file has about 10 columns and thousands of rows. The csv files look something like the following:

gene p-value xyz
acan 0.05 123
mmp2 0.02 456
mmp9 0.07 789
nnos 0.09 123
gfap 0.01 456

I have written the following script with the purpose of going through each file and filtering the rows only by the genes of interest that I have indicated (in this case mmp2 and mmp9).

# the goal is to edit and save the csv files so they only contain the genes of interest

path = '/Users/adriana/Library/Documents/raw_data',
all_files = glob.glob(os.path.join(path, "*.csv")) #make list of file paths 
genes = ["mmp2", "mmp9"]
for file in all_files:
    path = '/Users/adriana/Library/Documents/raw_data'
    df = pd.read_csv(file,delimiter ='\t')
    cleaned = df[df['gene'].isin(genes)]
    cleaned.to_csv(file)

However, I get the following error related to creation of the object "all_files":

TypeError: expected str, bytes or os.PathLike object, not tuple

I've used this script before seamlessly, so I am not sure what is going on.


Solution

  • try this:

    import os
    import glob
    import pandas as pd
    
    
    
    path = '/Users/adriana/Library/Documents/raw_data'  # Removed comma here
    all_files = glob.glob(os.path.join(path, "*.csv"))  # make list of file paths 
    genes = ["mmp2", "mmp9"]
    for file in all_files:
        df = pd.read_csv(file, delimiter='\t')
        cleaned = df[df['gene'].isin(genes)]  
        cleaned.to_csv(file, index=False)