Search code examples
statsmodelsmissing-datakalman-filterstate-space

Handle missing values with Stats Models Local Linear Trend model


I am using statsmodels to fit a Local Linear Trend state space model which inherits from the sm.tsa.statespace.MLEModel class using the code from the example in the documentation:

https://www.statsmodels.org/dev/examples/notebooks/generated/statespace_local_linear_trend.html

The state space model and Kalman filter should handle missing values naturally but when I add some null values the state space model outputs nulls. In another example in the docs, implementing SARIMAX it appears that missing data appears to be handled automatically:

https://www.statsmodels.org/dev/examples/notebooks/generated/statespace_sarimax_internet.html

Is there a way to handle missing values in the same way for a Local Linear Trend model?


Solution

  • Chad Fulton replied to the issue I raised on github:

    https://github.com/statsmodels/statsmodels/issues/7684

    The statespace models can indeed handle NaN values in the endog variable. I think the issue is that in this example code, the starting parameters are computed as:

    @property
    def start_params(self):
        return [np.std(self.endog)]*3
    

    To handle NaN values in the data, you'd want to replace this with:

    @property
    def start_params(self):
        return [np.nanstd(self.endog)]*3
    

    This worked.