Appending tuples to a Pandas dataframe and "cannot concatenate" - what am I doing wrong?

Question

I want to fill up a dataframe from a scraping loop. Say I created an empty DF with four columns.

df = pd.DataFrame(columns=['A','B','C','D'])

Then the loop to fill it looks like this:

for i in range:
  a = x['col1'][i]
  b = x['col2'][i]
  c = x['col3'][i]
  d = x['col4'][i]
  tup = (a,b,c,d)
  df = df.append(tup)

This is the error I get, though:

TypeError: cannot concatenate object of type '<class 'str'>'; only Series and DataFrame objs are valid

What am I doing wrong?

Answer 1

You can add dictionary instead of tuple:

df = pd.DataFrame(columns=["A", "B", "C", "D"])

for i in range(10):
    a = i + 1
    b = i + 2
    c = i + 3
    d = i + 4
    d = {"A": a, "B": b, "C": c, "D": d}
    df = df.append(d, ignore_index=True)

print(df)

Prints:

    A   B   C   D
0   1   2   3   4
1   2   3   4   5
2   3   4   5   6
3   4   5   6   7
4   5   6   7   8
5   6   7   8   9
6   7   8   9  10
7   8   9  10  11
8   9  10  11  12
9  10  11  12  13

But using .append like this is anti-pattern. You should see FutureWarning that this method will be removed in future version of pandas.

Instead, create dataframe in one go and skip creating empty dataframe at the beginning:

df = pd.DataFrame(
    [{"A": i + 1, "B": i + 2, "C": i + 3, "D": i + 4} for i in range(10)]
)