There is plenty of information describing Transformers in a lot of detail how to use them for NLP tasks. Transformers can be applied for time series forecasting. See for example "Adversarial Sparse Transformer for Time Series Forecasting" by Wu et al.

For understanding it is best to replicate everything according to already existing examples. There is a very nice example for LSTM with flights dataset https://stackabuse.com/time-series-prediction-using-lstm-with-pytorch-in-python/.

I guess I would like to know how to implement transformers for at first univariate (flight dataset) and later for multivariate time series data. What should be removed from the Transformer architecture to form a model that would predict time series?


There is an implementation of the paper ("Adversarial Sparse Transformer for Time Series Forecasting"), in Python using Pytorch, here. Although it has the training and evaluation functionality implemented, it appears to be lacking a function for running a prediction. Maybe you can fork it and extend it.


There is also a paper, "Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting", by Zhou et al., which does forecasts on univariate and multivariate data. Their code is here.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.