Search code examples
pythonstatsmodelsstate-space

Loading saved params (a Pandas series) into a Statsmodels state-space model


I'm building a dynamic factor model using the excellent python package statsmodels, and I would like to pickle an estimated parameter vector so I can build the model again later, and load those params into it. (C.f., this Notebook built by Chad Fulton: https://github.com/ChadFulton/tsa-notebooks/blob/master/dfm_coincident.ipynb.)

In the following block of code, initial parameters are estimated with mod.fit() (using the Powell algo) and then given back to mod.fit() to complete the estimation (using the EM algo) using the initial parameters as initial_res.params. (The latter is a Pandas Series.)

mod = sm.tsa.DynamicFactor(endog, k_factors=1, factor_order=2, error_order=2)
initial_res = mod.fit(method='powell', disp=False)
res = mod.fit(initial_res.params)

I would like to pickle res.params (again, a small Pandas Series, and a small disk footprint). Then later build the model from scratch again, and load my saved parameters into it without having to re-estimate the model. Anyone know how that can be done?

Examples I have seen suggest pickling the results object res, but that can be a pretty big save. Building it from scratch is pretty simple, but estimation takes a while. It may be that estimation starting from the saved optimal params is quicker; but still, that's pretty amateurish, right?

TIA, Drew


Solution

  • You can use the smooth method on any state space model to construct a results object from specific parameters. In your example:

    mod = sm.tsa.DynamicFactor(endog, k_factors=1, factor_order=2, error_order=2)
    initial_res = mod.fit(method='powell', disp=False)
    res = mod.fit(initial_res.params)
    
    res.params.to_csv(...)
    
    # ...later...
    
    params = pd.read_csv(...)
    
    res = mod.smooth(params)