Pandas - applying multi index labels in a loop

Question

I have an arbitrary number of DataFrames I would like to concatenate but keep an identifier on each DataFrame when they are all together. My problem is similar to Concatenate Pandas columns under new multi-index level.

The problem is as follows. I have multiple DataFrames like this;

data           Open     High      Low    Close   Volume
Date                                                   
2002-01-17  0.18077  0.18800  0.16993  0.18439  1720833
2002-01-18  0.18439  0.21331  0.18077  0.19523  2027866
2002-01-21  0.19523  0.20970  0.19162  0.20608   771149

And I would like to concatenate them to this;

symbol         ABC                                       XYZ
data           Open     High      Low    Close   Volume  Open ...
Date                                                   
2002-01-17  0.18077  0.18800  0.16993  0.18439  1720833  ...
2002-01-18  0.18439  0.21331  0.18077  0.19523  2027866  ...
2002-01-21  0.19523  0.20970  0.19162  0.20608   771149  ...

However I am doing this through a loop where I will generate the ABC or XYZ labels for that specific DataFrame. When concatenating the DataFrames I want to concat the new DataFrame with the generated ID. How can I do this?

The solution in the linked question applies the keys all at once but since I am concatenating in a loop (due to the arbitrary number of DataFrames) the solution does not apply in this case.

Thanks

Answer 1

You should still be able to use pd.concat in the same manner. Save all your DataFrames to a list and all the labels for those DataFrames in another list and then use 'pd.concat'. Here is some pseudocode that your code can mirror.

dfs = []
df_labels = []

# for loop that generates dataframes
for i in someiterable:
    dfs.append(df_new)
    df_labels.append(label_new)

pd.concat(dfs, keys=df_labels, axis=1)