There is plenty of information describing Transformers in a lot of detail how to use them for NLP tasks. Transformers can be applied for time series forecasting. See for example "Adversarial Sparse Transformer for Time Series Forecasting" by Wu et al.
For understanding it is best to replicate everything according to already existing examples. There is a very nice example for LSTM with flights dataset https://stackabuse.com/time-series-prediction-using-lstm-with-pytorch-in-python/.
I guess I would like to know how to implement transformers for at first univariate (flight dataset) and later for multivariate time series data. What should be removed from the Transformer architecture to form a model that would predict time series?