Search code examples
pythonmachine-learningtime-seriesdata-scienceforecasting

Can you forecast timeseries trend after lagging the target variable thus removing the trend?


Probably I'm missing something obvious--when I detrend my timeseries target data my model preforms way better. That's great. However, I'm trying to forecast an entire cycle and the trend ~is~ important. Is there a way to reconstitute the trend with these better scores or am I shooting in the foot by removing the trend in the first place?

mean absolute error with trend intact are on order of 0.001-0.003, with trend removed the scores are around 0.0001


Solution

  • Please provide more information. What kind of model do you use? Can you give an example of the time series e.g. pd.Series(data=[100,110,120,130,140])? Have you checked for overfitting, meaning your model performs good on your current dataset but once new data comes in it performs really poor. Does your time series really have a trend, or does it more or less move sideways (plot-wise speaking)?

    Also you can combine different models, for example a linear model model might be a good choice for simulating the trend. Once you implemented the linear trend model you can add another model which tries to predict where the linear trend model is wrong. So esentially you could add a random forest algorithm which predicts the residuals of the linear model. After you got both models you can simly sum up the prediction of both models. The linear one for the general trend and the random forest which tries to predict seasonality.

    You can also look into models which recognize seasonality by nature, such as ARIMA models for example.