I need to run a lot of OLS regressions (~1.600). I have collected 60 data points for each of the ~1.600 observations.
I am using the Fama & French 5 factor model, where the 60 data points for each of the observations is matched with the dates in the sample. E.g. I have the five factor parameters for a start date ['2010-1-1'] to an end date ['2015-1-1'] in a dataframe.
I need to run these parameters against stock returns for a given stock. Now, since the five factor parameters are collected in a dataframe, with around 96.000 rows (1600*60) and five columns (for each factor), I need to select the first 60 observations, run these against a set of returns with OLS, store the estimated coefficients and then select the next 60 observations for both the factor parameters and stock returns.
I have tried using slicing like:
start = 0
stop = 59
empty_list = []
for i in my_data:
coef = my_date[i][start:stop]
# run regression with the coef slice and store them in a dataframe
start += 60
stop += 60
However, I can't seem to get this to work. Any suggestions for how to solve this?
use groupby + np.arange() // 60
from statsmodels.api import formula
import pandas as pd
df = pd.DataFrame(
np.random.randn(96000, 6),
columns=['f1', 'f2', 'f3', 'f4', 'f5', 'r']
)
f = 'r ~ f1 + f2 + f3 + f4 + f5'
def regress(df, f):
return formula.ols(f, df).fit().params
results = df.groupby(np.arange(len(df)) // 60).apply(regress, f=f)
results.head()
Intercept f1 f2 f3 f4 f5
0 -0.108910 0.205059 0.006981 0.088200 0.064486 -0.003423
1 0.155242 -0.057223 -0.097207 -0.098114 0.163142 -0.029543
2 0.014305 -0.123687 -0.120924 0.017383 -0.168981 0.090547
3 -0.254084 -0.063028 -0.092831 0.137913 0.185524 -0.088452
4 0.025795 -0.126270 0.043018 -0.064970 -0.034431 0.081162