Search code examples
pythonpandasloopsnamedtuple

How to append row from itertuples to dataframe withouth losing the index in Python?


I have the following problem:

I have a DataFrame df which looks like this:

       eqpid  recpid   queuetime           trackintime         trackouttime
3723   A      XYZ      2017-01-01 03:14:58 2017-01-04 03:43:28 2017-01-04 03:43:33
...    ...    ...      ...                 ...                 ...

I am iterating through this DataFrame with itertuples() (I checked vectorization & .apply(), does not work here unfortunately). Now, beside other operations: I want to append the row (which is a namedtuple in my understanding) to another DataFrame with the same columns and keep the initial index, so it looks like this:

       eqpid  recpid   queuetime           trackintime         trackouttime
...    ...    ...      ...                 ...                 ...
3723   A      XYZ      2017-01-01 03:14:58 2017-01-04 03:43:28 2017-01-04 03:43:33

In theory, the code should look something like this:

temp_df = pd.DataFrame(columns=df.columns)
for row in df.itertuples():
    ...
    temp_df.append(row)

But this does not work, the temp_df remains empty. I also tried something like this:

temp_df = pd.DataFrame(columns=df.columns)
for row in df.itertuples():
    ...
    temp = pd.DataFrame([row],columns = row._fields)
    temp.set_index('Index', inplace = True)
    temp_df.append(temp)

But even though when I print temp it looks like:

Index  eqpid  recpid   queuetime           trackintime         trackouttime
3723   A      XYZ      2017-01-01 03:14:58 2017-01-04 03:43:28 2017-01-04 03:43:33

The temp_df remains empty. Can someone give me a hint what to do or what my mistake is?

Thank you all in advance!


Solution

  • Try to use 'iterrows' which returns rows as a tuple of index,series:

    for idx,ser in df.iterrows():
        ...
        temp_df = temp_df.append(ser)
    

    The series itself contains the index, too, so the index alignment works.