I have a dataset where for each country there is a name (string) and a multivariate time series (all integers).
I took the idea of using an Embedding layer from here. This way the model can learn how the time series for each country behaves.
But how to train it?
Currently I am using a sliding window method based on the book "Long Short-Term Memory Networks With Python" in chapter 3.4 is says the following (emphasis added):
Sequence prediction problems must be re-framed as supervised learning problems.
So the data is divided into weeks and the model uses one week to predict one day.
d1, d2, d3, d4, d5, d6, d7 -> d8 d2, d3, d4, d5, d6, d7, d8 -> d9 ...
This is not a seq2seq model as it only outputs one time step. I make predictions in a loop in order to get 21 days worth of forecasts. Ideally I would like to make the model produce X number of days worth of predictions e.g. 28 days.
After reading some research papers I learnt about Direct forecasting methods. Where the authors mention it is possible to directly predict a time step based on previous data without having to use the sliding window method.
This is also known as moving window scheme in forecasting, and requires substantial data augmentation. Such method is not necessary in an RNN thanks to its temporal nature.
From Wen, R. et al 2018
A brief summary of the ﬁndings is that the direct strategy generally has the smallest bias, and becomes superior for longer time series.
From Taieb, S. B. et al 2016
From these two papers it seems that using the Direct method is better than my current Sliding Window method.
My problem is that I don't know how to train this model without using the sliding window method.
An idea I had was to use the last 28 days as the test data then use the last 28 days of the training data as the sample output. Then the model would use one big sample to produce 28 forecasts but this seems to cause a few problems.
- The model will only be trained on one sample.
- Input data in the test will overlap with the train data.
- Most values in the training data are small compared to the target data.
This leads to a few questions:
- Does sequence prediction require supervised learning or is it possible to use unsupervised learning?
- Will the Embedding layer really help the model learn how each country behaves?
- How to train an Encoder-Decoder on direct forecasting, i.e. no sliding window?