Search code examples
pythonpandasdatetimeconcatenation

Concat two datetime indexed Dataframes

I am having a problem concating two Dataframes without loosing the date time index.

df1:

DateTime             Open     High     Low      Close              
2022-09-22 09:19:00  143.562  143.619  143.227  143.246

df2:

DateTime             Open     High     Low      Close              
2022-09-22 09:20:00  143.660  143.702  143.367  143.355

I am trying to concat with following code:

combined_data = pd.concat([previous_df, df_test], axis=0)

when I try and use concat I get following output:

    Unnamed: 0  DateTime    Open    High    Low Close
    1   1   2022-09-22 09:19:00 NaN 143.562 143.619 143.227 143.246
    2   2022-09-22 09:20:00 NaN NaN 143.098 143.098 142.683 142.698

So now every single Datetime index is created as a new index

I have also tried using join = inner/outer with the same results. Using ignore_index = true/false has the same results

EDIT TO QUESTION:

I found out that the problem exists because df1 is loaded from a csv and either while saving to the csv or loading from csv the index gets lost.

The question now is only how I can save and load the index correctly from a csv.

code for saving to csv:

df.to_csv('datatest.csv')

code for loading from csv:

previous_df = pd.read_csv('datatest.csv')

combined_data = pd.concat([previous_df, df_test], axis=0)

combined_data
Solution

  • Problem arose from reading the csv. Csv did not save index correctly and renamed it to unnamed which is why the two df were not able to concat correctly