Can you prevent automatic alphabetical order of df.append()?

Question

I am trying to append data to a log where the order of columns isn't in alphabetical order but makes logical sense, ex.

Org_Goals_1  Calc_Goals_1  Diff_Goals_1   Org_Goals_2 Calc_Goals_2 Diff_Goals_2 

I am running through several calculations based on different variables and logging the results through appending a dictionary of the values after each run. Is there a way to prevent the df.append() function to order the columns alphabetically?

Answer 1

Seems you have to reorder the columns after the append operation:

In [25]:
# assign the appended dfs to merged
merged = df1.append(df2)
# create a list of the columns in the order you desire
cols = list(df1) + list(df2)
# assign directly
merged.columns = cols
# column order is now as desired
merged.columns
Out[25]:
Index(['Org_Goals_1', 'Calc_Goals_1', 'Diff_Goals_1', 'Org_Goals_2', 'Calc_Goals_2', 'Diff_Goals_2'], dtype='object')

example:

In [26]:

df1 = pd.DataFrame(columns=['Org_Goals_1','Calc_Goals_1','Diff_Goals_1'], data = randn(5,3))
df2 = pd.DataFrame(columns=['Org_Goals_2','Calc_Goals_2','Diff_Goals_2'], data=randn(5,3))
merged = df1.append(df2)
cols = list(df1) + list(df2)
merged.columns = cols
merged
Out[26]:
   Org_Goals_1  Calc_Goals_1  Diff_Goals_1  Org_Goals_2  Calc_Goals_2  \
0     0.028935           NaN     -0.687143          NaN      1.528579   
1     0.943432           NaN     -2.055357          NaN     -0.720132   
2     0.035234           NaN      0.020756          NaN      1.556319   
3     1.447863           NaN      0.847496          NaN     -1.458852   
4     0.132337           NaN     -0.255578          NaN     -0.222660   
0          NaN      0.131085           NaN     0.850022           NaN   
1          NaN     -1.942110           NaN     0.672965           NaN   
2          NaN      0.944052           NaN     1.274509           NaN   
3          NaN     -1.796448           NaN     0.130338           NaN   
4          NaN      0.961545           NaN    -0.741825           NaN   

   Diff_Goals_2  
0           NaN  
1           NaN  
2           NaN  
3           NaN  
4           NaN  
0      0.727619  
1      0.022209  
2     -0.350757  
3      1.116637  
4      1.947526  

The same alpha sorting of the columns happens with concat also so it looks like you have to reorder after appending.

EDIT

An alternative is to use join:

In [32]:

df1.join(df2)
Out[32]:
   Org_Goals_1  Calc_Goals_1  Diff_Goals_1  Org_Goals_2  Calc_Goals_2  \
0     0.163745      1.608398      0.876040     0.651063      0.371263   
1    -1.762973     -0.471050     -0.206376     1.323191      0.623045   
2     0.166269      1.021835     -0.119982     1.005159     -0.831738   
3    -0.400197      0.567782     -1.581803     0.417112      0.188023   
4    -1.443269     -0.001080      0.804195     0.480510     -0.660761   

   Diff_Goals_2  
0     -2.723280  
1      2.463258  
2      0.147251  
3      2.328377  
4     -0.248114