I would like to extend a Panel frame of data along a minor axis in pandas. I start off creating a dic of DataFrames to generate a Panel.
import pandas as pd
import numpy as np
rng = pd.date_range('1/1/2013',periods=100,freq='D')
df1 = pd.DataFrame(np.random.randn(100, 4), index = rng, columns = ['A','B','C','D'])
df2 = pd.DataFrame(np.random.randn(100, 4), index = rng, columns = ['A','B','C','D'])
df3 = pd.DataFrame(np.random.randn(100, 4), index = rng, columns = ['A','B','C','D'])
pf = pd.Panel({'df1':df1,'df2':df2,'df3':df3})
As expected I, find I have a panel with the following dimensions:
Dimensions: 3 (items) x 100 (major_axis) x 4 (minor_axis) Items axis: df1 to df3 Major_axis axis: 2013-01-01 00:00:00 to 2013-04-10 00:00:00 Minor_axis axis: A to D
I would now like to add a new data set to the Minor axis:
pf['df1']['E'] = pd.DataFrame(np.random.randn(100, 1), index = rng)
pf['df2']['E'] = pd.DataFrame(np.random.randn(100, 1), index = rng)
pf['df2']['E'] = pd.DataFrame(np.random.randn(100, 1), index = rng)
I find that after adding this new minor axis the shape of the panel array dimensions has not changed:
shape(pf)
[3,100,4]
I am able to access the data for each of the items in the major_axis:
pf.ix['df1',-10:,'E']
2013-04-01 0.168205 2013-04-02 0.677929 2013-04-03 0.845444 2013-04-04 0.431610 2013-04-05 0.501003 2013-04-06 -0.403605 2013-04-07 -0.185033 2013-04-08 0.270093 2013-04-09 1.569180 2013-04-10 -1.374779 Freq: D, Name: E
But if I extend the slicing to include more than one major axis:
pf.ix[:,:,'E']
Then I encounter an error saying that 'E' is unknown.
Can anyone suggest where I am going wrong or a better way of performing this operation?
This doesn't work right now see this, https://github.com/pydata/pandas/issues/2578 But you can accomplish what you want this way. This is a pretty cheap operation as nothing is copied.
In [18]: x = pf.transpose(2,0,1)
In [19]: x
Out[19]:
<class 'pandas.core.panel.Panel'>
Dimensions: 4 (items) x 3 (major_axis) x 100 (minor_axis)
Items axis: A to D
Major_axis axis: df1 to df3
Minor_axis axis: 2013-01-01 00:00:00 to 2013-04-10 00:00:00
In [20]: x['E'] = new_df
In [21]: x.transpose(1,2,0)
Out[21]:
<class 'pandas.core.panel.Panel'>
Dimensions: 3 (items) x 100 (major_axis) x 5 (minor_axis)
Items axis: df1 to df3
Major_axis axis: 2013-01-01 00:00:00 to 2013-04-10 00:00:00
Minor_axis axis: A to E