Search code examples
pythonpandasdataframemulti-level

Modify slice of multiindex with series


I have a problem that I have been trying to wrap my head around for most of the day, so far in vain.

I want to:

  1. take a series from a multi(column) index
  2. modify it with a mathematical operator, or an .apply() function
  3. return it to the dataframe.

I can do 1 and 2, but for some reason i can't properly return it to the dataframe. I would also be happy to construct a new dataframe from the series. By this point, I am willing to do many things to solve it.

Example data looks like this:

Foo Bar Baz
day night day night day night
a a b b a a b b a a b b a a b b a a b b a a b b
12 5 33 50 12 5 33 50 12 5 33 50 12 5 33 50 12 5 33 50 12 5 33 50
id
1 0 2 1 0 12 0 18 0 9 0 3 0 2 0 0 0 0 15 0 0 3 0 0 0
2 1 4 0 0 0 0 0 0 0 2 7 0 0 2 0 11 1 19 0 0 0 6 1 0
3 0 0 0 3 20 1 0 14 0 3 2 0 3 0 0 0 1 0 0 16 0 1 6 0
4 0 3 2 0 0 10 0 0 0 2 0 12 0 0 20 4 5 0 0 0 1 0 0 0
5 16 0 0 0 7 2 0 0 0 0 0 0 18 1 0 0 0 0 5 0 5 13 0 0
6 0 0 10 6 1 0 9 13 0 3 8 0 4 8 0 0 11 14 0 0 21 0 0 0
7 7 0 0 0 0 19 0 0 1 17 0 0 0 0 22 0 2 0 0 0 0 2 0 17

I can get the data by some variant of:

in:
df.loc[2].loc['Foo','day']
out:
a  12    1
   5     4
b  33    0
   50    0

Now I want to do something along the lines of:

df.loc[2].loc['Foo','day'] == df.loc[2].loc['Foo','day'] * (60*24)

and have the dataframe updated with the new data. It is to do iterative editing later on. But for some reason nothing changes my dataframe. I've tried slicing it and updating in all the different ways I can think of.

I've tried the code in jezrael's answer to jaydeepsb here: Pandas update values in a multi-index dataframe Which tought me to update a slice with a value, but so far updating a slice with a series eludes me.

If anyone can help me, I thank you greatly. Also for not making me throw my computer out of the window in frustration :D


Solution

  • (1) Locate the subset of your dataframe, (2) apply your operation and (3) update your dataframe:

    idx = pd.IndexSlice  # for multi-index slicing
    
    df.update(df.loc[[2], idx["Foo", "day", :, :]] * (60*24))