Search code examples
pythonsktime

Inconsistant prediction range settings in sktime

I noticed an inconsistency in specifying the prediction intervals for different kind of algorithms - AutoETS and AutoARIMA. I'm not sure if this is a bug or a feature.

from matplotlib import pyplot as plt

from sktime.datasets import load_airline
from sktime.forecasting.model_selection import temporal_train_test_split
from sktime.forecasting.base import ForecastingHorizon
y = load_airline()
y_train,y_test = temporal_train_test_split(y)
fh = ForecastingHorizon(y_test.index, is_relative=False)

from sktime.forecasting.ets import AutoETS
model = AutoETS(trend='add',seasonal='mul',sp=12)
model.fit(y_train,fh=y_test.index)
preds_ets_05 = model.predict(fh,return_pred_int=True,alpha=0.05)
preds_ets_95 = model.predict(fh,return_pred_int=True,alpha=0.95)

from sktime.forecasting.arima import AutoARIMA
model = AutoARIMA(tsp=12)
model.fit(y_train,fh=y_test.index)
preds_arima_05 = model.predict(fh,return_pred_int=True,alpha=0.05)
preds_arima_95 = model.predict(fh,return_pred_int=True,alpha=0.95)

If we plot the forecasts, we get:

figs, (ax1,ax2) = plt.subplots(2,sharey=True)
ax1.fill_between(preds_ets_05[0].index.to_timestamp('M'),
                 preds_ets_05[1]['lower'],
                 preds_ets_05[1]['upper'],
                 alpha=0.25,
                 color='green',
                 label = 'ets')
ax1.fill_between(preds_arima_05[0].index.to_timestamp('M'),
                 preds_arima_05[1]['lower'],
                 preds_arima_05[1]['upper'],
                 alpha=0.25,
                 color='red',
                 label ='arima')
ax1.tick_params(rotation=45)
ax1.set_title('alpha=0.05')
ax1.legend()

ax2.fill_between(preds_ets_95[0].index.to_timestamp('M'),
                 preds_ets_95[1]['lower'],
                 preds_ets_95[1]['upper'],
                 alpha=0.25,
                 color='green',
                 label = 'ets')
ax2.fill_between(preds_arima_95[0].index.to_timestamp('M'),
                 preds_arima_95[1]['lower'],
                 preds_arima_95[1]['upper'],
                 alpha=0.25,
                 color='red',
                 label = 'arima')
ax2.tick_params(rotation=45)
ax2.set_title('alpha=0.95')
ax2.legend()
plt.tight_layout()
plt.show()

enter image description here

It looks like the definition of alpha is reversed for one of the algos.

Solution

  • Known bug in version 0.10.X (intervals should get wider as coverage gets larger), should be fixed in 0.11.0, see https://github.com/alan-turing-institute/sktime/discussions/2334