Search code examples
rtime-seriesforecasting

R: Why does it mean when the MASE of a forecast model is NaN?

See this monthly data:

tsdata <- structure(c(9.55584, 42.31872, 17.064, 54.26352, 79.51824, 44.3664, 
                      82.58976, 129.6864, 70.64496, 102.384, 118.08288, 99.31248, 151.8696, 
                      172.68768, 129.34512), .Tsp = c(2015.25, 2016.41666666667, 12
                      ), class = "ts")

plot(tsdata)

enter image description here

Then, I split the series into training and test sets:

training <- structure(c(9.55584, 42.31872, 17.064, 54.26352, 79.51824, 44.3664, 
                        82.58976, 129.6864, 70.64496, 102.384, 118.08288, 99.31248),
                      .Tsp = c(2015.25, 2016.16666666667, 12), class = "ts")

test <- structure(c(151.8696, 172.68768, 129.34512), .Tsp = c(2016.25, 2016.41666666667, 12), class = "ts")

Finally, I fit a tbats model and calculate the MASE value

require(forecast)
fit <- tbats(training)
fcast <- forecast(fit)
accuracy(fcast, test)
                     ME     RMSE      MAE       MPE     MAPE MASE       ACF1 Theil's U
Training set -0.9012629 20.02206 16.33986 -22.70125 40.40976  NaN -0.3870826        NA
Test set     12.3136351 25.58155 24.77819   6.50544 16.14211  NaN -0.2992376  0.860442

You see that MASE could not be calculated. Actually, all the other models from the forecast package also outputs NaN for MASE.

Why? The MASE metric is really important to me and it is the one that I want to use for comparing several models for my time series data.

Solution

  • The MASE uses a scaling factor computed on the training data. For seasonal data, the default scaling factor is the average of the absolute seasonal differences. With only one year of data, you cannot compute seasonal differences, so the scaling factor is NaN.

    You could use a scaling factor based on first differences instead as follows:

    accuracy(fcast, test, d=1, D=0)

    However, I just noticed that there is a bug in accuracy() which means the d and D arguments are being ignored. It is now fixed and the version on github will use the arguments specified.