Search code examples
pythonpandasdataframeloopsrow

How to Pivot Columns of a Pandas DataFrame into Inner-most Level Index without Using df.iterrows()?


The original .csv file -

#,Name,Type 1,Type 2,HP,Attack,Defense,Sp. Atk,Sp. Def,Speed,Generation,Legendary
1,Bulbasaur,Grass,Poison,45,49,49,65,65,45,1,FALSE
2,Ivysaur,Grass,Poison,60,62,63,80,80,60,1,FALSE
3,Venusaur,Grass,Poison,80,82,83,100,100,80,1,FALSE

My Python code using df.iterrows() -

import pandas as pd
import os

df = pd.read_csv('pokemon_data.csv')
with open('output.txt', 'w') as f:
    for index, row in df.iterrows():
        row_i = str(index) + str(row)
        f.write(row_i)

I learned that we should avoid using df.iterrow(), for it would get very slow when dealing with big data.

How can I pivot the columns of a Pandas DataFrame into the inner-most level index, and get the result as follows, without using df.iterrows(), then?

0 #                     1
Name          Bulbasaur
Type 1            Grass
Type 2           Poison
HP                   45
Attack               49
Defense              49
Sp. Atk              65
Sp. Def              65
Speed                45
Generation            1
Legendary         False

1 #                   2
Name          Ivysaur
Type 1          Grass
Type 2         Poison
HP                 60
Attack             62
Defense            63
Sp. Atk            80
Sp. Def            80
Speed              60
Generation          1
Legendary       False

2 #                    3
Name          Venusaur
Type 1           Grass
Type 2          Poison
HP                  80
Attack              82
Defense             83
Sp. Atk            100
Sp. Def            100
Speed               80
Generation           1
Legendary        False

Solution

  • We can try stack + to_string:

    df.stack().to_string('output.txt')
    

    output.txt:

    0  #                     1
       Name          Bulbasaur
       Type 1            Grass
       Type 2           Poison
       HP                   45
       Attack               49
       Defense              49
       Sp. Atk              65
       Sp. Def              65
       Speed                45
       Generation            1
       Legendary         False
    1  #                     2
       Name            Ivysaur
       Type 1            Grass
       Type 2           Poison
       HP                   60
       Attack               62
       Defense              63
       Sp. Atk              80
       Sp. Def              80
       Speed                60
       Generation            1
       Legendary         False
    2  #                     3
       Name           Venusaur
       Type 1            Grass
       Type 2           Poison
       HP                   80
       Attack               82
       Defense              83
       Sp. Atk             100
       Sp. Def             100
       Speed                80
       Generation            1
       Legendary         False