Search code examples
pythonpandasindexingseriesmulti-index

Setting values in a pandas MultiIndex series


I'm trying to set a series into another one, at a multi-index value. I cannot find a way to do it in Pandas without complex hacks.

My original series:

one  1    0.522764
     3    0.362663
     7    0.963108
two  2    0.717855
     4    0.004645
     5    0.077471

The data I want to concatenate, at level three:

2    0.8
7    0.9
8    0.7

The desired output:

one    1    0.522764
       3    0.362663
       7    0.963108
two    2    0.717855
       4    0.004645
       5    0.077471
three  2    0.800000
       7    0.900000
       8    0.700000

I cannot figure out an elegant way to do this in pandas. All I've been able to do is the following hack:

# imports
import numpy as np
import pandas as pd 

# to replicate the Series: 
np.arrays = [['one','one','one','two','two','two'],[1,3,7,2,4,5]]
my_series = pd.Series([np.random.random() for i in range(6)],
               index=pd.MultiIndex.from_tuples(list(zip(*np.arrays))))

# the new data I need to add: 
new_data = pd.Series({1: .9, 2: .7, 3: .8})

Here is how I'm currently solving it:

# rename the index so that I can call it later 
new_data.index.name = 'level_1' 

# turn it into temporary a dataframe so that I can add a new column 
temp = pd.DataFrame(new_data) 

# create a new column with the desired name for first index level 
temp['level_0'] = 'three'   

# reset index, set the new index, turn into Series again
temp = temp.reset_index().set_index(['level_0', 'level_1'])[0]                              

# append it to the larger dataframe 
my_series = my_series.append(temp)                  

This yields the desired output.

Question: Is there a simple, elegant way to do this in Pandas?


Solution

  • Option 1

    pd.concat is a handy way to prepend an index or column level by using the keys argument. Combined this with a second pd.concat to get the job done.

    pd.concat([my_series, pd.concat([new_data], keys=['Three'])])
    
    one    1    0.943246
           3    0.412200
           7    0.379641
    two    2    0.883960
           4    0.182983
           5    0.773227
    Three  1    0.900000
           2    0.700000
           3    0.800000
    dtype: float64
    

    Option 2
    Or we can construct a new series while inserting an additional array into the index argument. Using pd.concat again to combine. Note I could have used pd.MultiIndex.from_arrays but the syntax is simplified by just passing the arrays directly to the index argument.

    pd.concat([
        my_series,
        pd.Series(new_data.values, [['Three'] * new_data.size, new_data.index])
    ])
    
    one    1    0.943246
           3    0.412200
           7    0.379641
    two    2    0.883960
           4    0.182983
           5    0.773227
    Three  1    0.900000
           2    0.700000
           3    0.800000
    dtype: float64
    

    Option 3
    Yet another way to reconstruct a series with a multiindex. This one uses pd.MultiIndex.from_product.

    pd.concat([
        my_series,
        pd.Series(new_data.values, pd.MultiIndex.from_product([['Three'], new_data.index]))
    ])
    
    one    1    0.943246
           3    0.412200
           7    0.379641
    two    2    0.883960
           4    0.182983
           5    0.773227
    Three  1    0.900000
           2    0.700000
           3    0.800000
    dtype: float64