Pandas raise KeyError(key) from err KeyError:, even though the key exists

Question

I am trying to create a script to iterate over .csv files in a folder, and then run some calculations on them, before saving to a new .csv file. I have been able to get this to work fine when producing means and percentages, but there's the KeyError issue when trying to add in some conditions.

Here is the code I have produced so far:

import os
import pandas as pd
import csv


folder_path = 'D:/Libraries/Documents/data'


rtAvg = 'block1_respo.rt'
cond = 'block1_trigger'

output_file = 'results_compiled.csv'


df_list = []


for filename in os.listdir(folder_path):
    if filename.endswith('.csv'):
        # Load the CSV file into a data frame
        df = pd.read_csv(os.path.join(folder_path, filename))
    # This code will check if the column is missing, and will populate that row with "9999" if it is
        if rtAvg not in df.columns:
            df[rtAvg] = 9999
        
       
        rt_mean_b1 = df[rtAvg].mean()
        

        con1 = df.loc[df[cond]==101][rtAvg].mean()
        con2 = df.loc[df[cond]==102][rtAvg].mean()
        con3 = df.loc[df[cond]==103][rtAvg].mean()
        con4 = df.loc[df[cond]==104][rtAvg].mean()
    

    # Create a new row for the summary data
        summary_row = pd.DataFrame({
            'csv_file': filename,
            'rt_average': rt_mean_b1,
            '101_rt':con1,
            '102_rt':con2,
            '103_rt':con3,
            '104_rt':con4
        },index = [0])
    
    # Append the summary data to the list of data frames
        df_list.append(summary_row)
    

summary_df = pd.concat(df_list)


summary_df.to_csv( output_file, index = False)

Here is the error code I am receiving:

con1 = df.loc[df[cond]==101][rtAvg].mean()
  File "C:\Program Files\PsychoPy\lib\site-packages\pandas\core\frame.py", line 3458, in __getitem__
    indexer = self.columns.get_loc(key)
  File "C:\Program Files\PsychoPy\lib\site-packages\pandas\core\indexes\base.py", line 3363, in get_loc
    raise KeyError(key) from err
KeyError: 'block1_trigger'

block1_trigger definitely exists. I have printed the lists and it is in there. I had also previously tried removing the white space with code but it made no difference. I also tried cond = ' block1_trigger' and 'block1_trigger ' and neither produced the expected results.

Answer 1

I have figured out the error. Some of the files were missing this column. I have now added a catch code to sort it out.

if cond not in df.columns:
        df[cond] = 9999