Suppose one has time series (univariate or multivariate) and the goal is to predict values of these series several steps ahead. I see two possible strategies:
Create a model (recurrent, convolutional, transformer, whatever) that predicts the value of the signal in the next moment of time, based on the values from previous timestamps from
(t_start, t_end). If we aim to predict not one, but several steps ahead we can pass
(signal[t_start + 1: t_end], signal[t_end + 1])to predict
signal[t_end + 2]and so on, so on. In the training stage we can pass predicted value of
signal[t_end + 1]or the ground truth with some probability, this can be seen as some kind of teacher forcing. In the inference stage, one passes each time the predicted signal. The optimization algorithm aims to minimize (MSE, MAE) loss between the ground truth and prediction.
Create model that predicts simultaneously several values ahead. Standard layers from DL frameworks (PyTorch of Tensorflow) for sequence processing problems have two options - output single hidden state in the end or the whole sequence of the hidden states. Therefore, seems like they do not have functionality, say, to predict values of the time series 16 steps ahead from the values of last 256 timestamps.
I see two potential solutions:
- output hidden state (16) times larger then the expected output and reshape - however, it seems that this approach breaks the locality and causal structure and would not achieve good performance.
- Choose the option, that returns the sequence of the same length as the input (here 256) and take the last (16) tokens of the output. This approach is inapplicable if the length of the prediction exceeds the length of the previous history, but I think, that such long predictions would produce poor quality in any case.
How stock, weather, sales prediction problems are solved usually in practice?