Search code examples
pythonpandasconcatenation

pd.concat not stacking columns with same names

I'm in awe that this isn't working because I've done this a hundred times. I want to stack two dataframes vertically and pandas is creating duplicate columns and refusing to put the data in the right columns.

df1 looks like this:

enter image description here

df2 looks like this:

enter image description here

then I run this:

frames = [df1,df2]
final = pd.concat(frames, ignore_index = True, axis = 0)
final

and get 6 columns instead of 3 like this:

enter image description here

I have no idea why two dataframes with identical column names and data types would not simply stack on top of each other. Any help appreciated.

Thanks.

update:

Big Thanks to @Naveed there was trailing whitespace in one of the dataframe's columns.

Here is the final fix:

df2.columns = [x.strip() for x in df2.columns]
frames = [df1,df2]
final = pd.concat(frames,ignore_index = True, axis = 0)
final
Solution

  • Try

    check the column names, there might be white spaces that results in mis-alignment of column after the concat.

    display(df1.columns, df2.columns)

    #  make axis=0 and remove ignore_index

final = pd.concat(frames,   axis = 0)
final