How to merge single-column pandas data frames in Python?

Question

I want to merge the dataset into a 1432 rows x 4 columns data frame. After I used for loop function to filter the all data, the output was separated into 4 outputs, each 1432 rows x 1 column. However, I want them to merge into one table. How I can merge them?

My code and its output:

for ind,row in gf.iterrows():
    filter2 = savgol_filter(row, 31,3)
    hf = pd.DataFrame(filter2)
    hf.to_numpy()
  
    print(hf)

Output:

             0
0     0.901141
1     0.915138
2     0.928173
3     0.940281
4     0.951494
...        ...
1427  0.108484
1428  0.111043
1429  0.113958
1430  0.117230
1431  0.120859

[1432 rows x 1 columns]
             0
0     0.926357
1     0.940313
2     0.953292
3     0.965326
4     0.976451
...        ...
1427  0.108484
1428  0.111043
1429  0.113958
1430  0.117230
1431  0.120859

[1432 rows x 1 columns]
             0
0     0.926577
1     0.941009
2     0.954399
3     0.966784
4     0.978202
...        ...
1427  0.108484
1428  0.111043
1429  0.113958
1430  0.117230
1431  0.120859

[1432 rows x 1 columns]
             0
0     0.928050
1     0.942212
2     0.955387
3     0.967608
4     0.978907
...        ...
1427  0.108484
1428  0.111043
1429  0.113958
1430  0.117230
1431  0.120859

Answer 1

In absence of knowledge of what savgol_filter() does, reformulating your for loop into a comprehension may be the best guess:

hf = pd.concat([pd.DataFrame(savgol_filter(row, 31, 3)) for (ind, row) in gf.iterrows()], axis=1)

Alternatively:

hf = pd.concat([pd.DataFrame(savgol_filter(df.iloc[ix], 31, 3)) for ix in gf.index], axis=1)

Enforcing index ind to land as column name:

pd.concat([pd.DataFrame(columns = [ind],
                        data = row) for (ind, row) in df.iterrows()], axis=1)

Explanation

There are several flaws in your loop design:

• Your for loop assigns a new data frame to the same variable at every iteration and this does not look like something you want to do. Instead, the comprehension processes all rows as a list, which lets you concatenate them as one object in the end.
• .to_numpy does not look useful, since you request a data frame as final product. It also is not assigned to a variable so it is without purpose a in the loop.

Depending on the nature of your savgol_filter function, simpler syntaxes may be possible.