Suppose one has a time series (univariate or multivariate) and the goal is to predict values of these series several steps ahead. I see two possible strategies:

  1. Create a model (recurrent, convolutional, transformer, whatever) that predicts the value of the signal in the next moment of time, based on the values from previous timestamps from (t_start, t_end). If we aim to predict not one, but several steps ahead we can pass (signal[t_start + 1: t_end], signal[t_end + 1]) to predict signal[t_end + 2] and so on. In the training stage, we can pass the predicted value of signal[t_end + 1] or the ground truth with some probability, this can be seen as some kind of teacher forcing. In the inference stage, one passes each time the predicted signal. The optimization algorithm aims to minimize (MSE, MAE) loss between the ground truth and prediction. In other words $$ \begin{aligned} x_{t+1} &= f(x_t, \ldots, x_{t-N+1}) \\ x_{t+2} &= f(x_{t+1}, \ldots, x_{t-N+2}) \\ x_{t+k} &= f(x_{t+1}, \ldots, x_{t-N+k}) \\ \end{aligned} $$

  2. Create a model that predicts simultaneously several values ahead. Standard layers from DL frameworks (PyTorch of Tensorflow) for sequence processing problems have two options - output single hidden state in the end or the whole sequence of the hidden states. Therefore, seems like they do not have the functionality, say, to predict values of the time series 16 steps ahead from the values of the last 256 timestamps. $$ [y_{t+k}, \ldots, y_{t+1}] = f(x_t, \ldots x_{t - N + 1}) $$ I see two potential solutions:

    • output hidden state (16) times larger than the expected output and reshape - however, it seems that this approach breaks the locality and causal structure and would not achieve good performance.
    • Choose the option, that returns the sequence of the same length as the input (here 256) and take the last (16) tokens of the output. This approach is inapplicable if the length of the prediction exceeds the length of the previous history, but I think, that such long predictions would produce poor quality in any case.

How stock, weather, sales prediction problems are solved usually in practice?


1 Answer 1


I have found nice tutorial in the Tensorflow documentation: https://www.tensorflow.org/tutorials/structured_data/time_series

They implement and test both strategies.

  1. In the first case, for multi dimensional time series, they output the vector of dimension out_steps * series_dim and then reshape to (out_steps, series dim)

  2. They create a model (AR LSTM), that predicts one step ahead and then apply it several times, where the first step from the previous input is discarded, and the new prediction is last step in new input.

enter image description here

Second approach, seems to require less params, but the obtained quality for this specific case, seems to be comparable for both cases.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .