Search code examples
pythonpandasdataframesubtraction

Pandas DataFrame subtract a series using groupby classification


I have a DataFrame (Main) as below. The columns have a group classification as described in the Group Dict. There is a second DataFrame with Group values. I need to subtract from each column in Main the value from the corresponding group in Group Value DataFrame. The resulting table is shown below as well. (Exp: Main["AAA"]-Group_Value["Group1"], etc.) Is there a matrix form of doing this or will I need to use a for-loop?

Code:

import random
df = pd.DataFrame(index=pd.date_range("1/1/2018","1/10/2018"), columns= 
["AAA","BBB","CCC","DDD"])
df["AAA"]=range(100,110)
df["BBB"]=range(200,210)
df["CCC"]=range(300,310)
df["DDD"]=range(400,410)

Group_Dict = dict({"AAA":"Group1",  "BBB":"Group2", "CCC":"Group1", "DDD":"Group2"})

group_value = pd.DataFrame(index=pd.date_range("1/1/2018","1/10/2018"), columns=["Group1","Group2"])
group_value["Group1"]=range(10,29)[::2]
group_value["Group2"]=range(100,600)[::50]

## I need to do the following AAA-Group1, BBB-Group2 , CCC-Group1, DDD-Group2 

'enter image description here


Solution

  • Edit to match question update:

    df = pd.DataFrame(index=pd.date_range("1/1/2018","1/10/2018"), columns= 
    ["AAA","BBB","CCC","DDD"])
    df["AAA"]=range(100,110)
    df["BBB"]=range(200,210)
    df["CCC"]=range(300,310)
    df["DDD"]=range(400,410)
    
    Group_Dict = dict({"AAA":"Group1",  "BBB":"Group2", "CCC":"Group1", "DDD":"Group2"})
    
    group_value = pd.DataFrame(index=pd.date_range("1/1/2018","1/10/2018"), columns=["Group1","Group2"])
    group_value["Group1"]=range(10,29)[::2]
    group_value["Group2"]=range(100,600)[::50]
    
    sub_group = group_value.reindex(Group_Dict.values(), axis=1)\
                           .set_axis(Group_Dict.keys(), axis=1, inplace=False)
    
    df_out = (df - sub_group).reset_index()
    print(df_out)
    

    Output:

           index  AAA  BBB  CCC  DDD
    0 2018-01-01   90  100  290  300
    1 2018-01-02   89   51  289  251
    2 2018-01-03   88    2  288  202
    3 2018-01-04   87  -47  287  153
    4 2018-01-05   86  -96  286  104
    5 2018-01-06   85 -145  285   55
    6 2018-01-07   84 -194  284    6
    7 2018-01-08   83 -243  283  -43
    8 2018-01-09   82 -292  282  -92
    9 2018-01-10   81 -341  281 -141
    

    Original Answer before question update.

    Let's try this:

        main = pd.DataFrame({'Date':pd.date_range('01-01-2018',periods=10,freq='D'),
                             'AAA':np.arange(100,110),'BBB':np.arange(200,210),
                             'CCC':np.arange(300,310),'DDD':np.arange(400,410)})
        groupdict=pd.DataFrame({'Key':['AAA','BBB','CCC','DDD'],
                                'Group':['Group1','Group1','Group2','Group2']})
        groupvalue=pd.DataFrame({'Date':pd.date_range('01-01-2018',periods=10,freq='D'),
                                 'Group1':np.arange(10,29,2),'Group2':np.arange(100,575,50)})
        
        groupvalue=groupvalue.set_index('Date')
    
    main = main.set_index('Date')
    
    #Use reindex and set_axis to expand and match your main dataframe columns
    sub_group = groupvalue.reindex(groupdict.Group,axis=1)\
                          .set_axis(groupdict.Key, axis=1, inplace=False)
    
    #Subtract letting pandas handle data alighnment with indexes.
    df_out = (main - sub_group).reset_index()
    print(df_out)
    

    Output:

            Date  AAA  BBB  CCC  DDD
    0 2018-01-01   90  190  200  300
    1 2018-01-02   89  189  151  251
    2 2018-01-03   88  188  102  202
    3 2018-01-04   87  187   53  153
    4 2018-01-05   86  186    4  104
    5 2018-01-06   85  185  -45   55
    6 2018-01-07   84  184  -94    6
    7 2018-01-08   83  183 -143  -43
    8 2018-01-09   82  182 -192  -92
    9 2018-01-10   81  181 -241 -141