Pandas Concat - not permanent / in place?

Question

I wrote a small function to add 1 row to the end of a given DF. In my jupyter notebook file, the output window adds 1 row, and all appears to work perfectly as expected.

def add_row(df):

    df_temp_row = pd.DataFrame({'time':[np.nan], 'mid_o':[np.nan], 'mid_h':[np.nan], 'mid_l':[np.nan], 'mid_c':[np.nan]})
    df = pd.concat([df, df_temp_row], ignore_index=True, axis=0, copy=True)

    return df

However, when I check the df len(df.index) the result is 1 less row than expected, as if the function failed.

It seems odd. Any advice please?

Answer 1

Your code works well if you think to override the dataframe:

df = pd.DataFrame()
print('Original length:', len(df))
df = add_row(df)  # <- don't forget to reassign, the modification is not inplace
print('First iteration:', len(df))
df = add_row(df)
print('Second iteration:', len(df))
df = add_row(df)
print('Third iteration:', len(df))

Output:

Original length: 0
First iteration: 1
Second iteration: 2
Third iteration: 3

Dataframe after 3 iterations:

>>> df
   time  mid_o  mid_h  mid_l  mid_c
0   NaN    NaN    NaN    NaN    NaN
1   NaN    NaN    NaN    NaN    NaN
2   NaN    NaN    NaN    NaN    NaN

Another way to append a row with loc:

df.loc[len(df)] = np.nan
print(df)

# Output
   time  mid_o  mid_h  mid_l  mid_c
0   NaN    NaN    NaN    NaN    NaN
1   NaN    NaN    NaN    NaN    NaN
2   NaN    NaN    NaN    NaN    NaN
3   NaN    NaN    NaN    NaN    NaN