Search code examples
pythonpython-3.xpandasmergeconcatenation

Concatenating two dfs in Pandas


I wanted to know if there was a way to verify if a value in one df that was being Concatenated with another existed in both dfs or the other df as well prior to stacking them on top of each other??

I had two dfs as

df = pd.concat([df1, df2])
df = df.sort_values(by=['id', 'timestamp']).reset_index(drop=True)
df

   id  timestamp
0   1 1959-06-01
1   1 2019-01-01
2   1 2019-01-02
3   2 1989-12-01
4   2 2019-01-15
5   3 1999-01-25
6   3 2019-01-17
7   3 2019-02-01
8   3 2019-02-03

Was there a way to to verify that the I.D in df1 existed in df2 before concatenating similar to a merge? I didn't need to merge but instead concatenate the dfs on top of each other.

One df has multiple ids and timestamps and the other had only one ID and I wanted to make sure only IDs that existed in both were in the resulting concatenated df

thanks!


Solution

  • One workaround you can do here is to create a dummy column:

    df1["df"] = 1
    df2["df"] = 2
    df = pd.concat([df1, df2])
    

    That way you can see where each row was derived.