Skip to main content
Search type Search syntax
Tags [tag]
Exact "words here"
Author user:1234
user:me (yours)
Score score:3 (3+)
score:0 (none)
Answers answers:3 (3+)
answers:0 (none)
isaccepted:yes
hasaccepted:no
inquestion:1234
Views views:250
Code code:"if (foo != bar)"
Sections title:apples
body:"apples oranges"
URL url:"*.example.com"
Saves in:saves
Status closed:yes
duplicate:no
migrated:no
wiki:no
Types is:question
is:answer
Exclude -[tag]
-apples
For more details on advanced search visit our help page
Results tagged with
Search options answers only not deleted user 5763

For questions related to the transformer, which is a deep machine learning model introduced in 2017 in the paper "Attention Is All You Need", used primarily in the field of natural language processing (NLP).

6 votes

How do open source LLMs compare to GPT-4?

The remarkable performance of GPT 4 is due to the massive size of its architecture and the amount of data it was trained on, which costs a lot of money. Few organizations have the hardware resources a …
Brian O'Donnell's user avatar
1 vote
Accepted

How to construct Transformers to predict multidimensional time series?

There is an implementation of the paper ("Adversarial Sparse Transformer for Time Series Forecasting"), in Python using Pytorch, here. … UPDATE There is also a paper, "Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting", by Zhou et al., which does forecasts on univariate and multivariate data. …
Brian O'Donnell's user avatar
5 votes

What kind of word embedding is used in the original transformer?

No, neither Word2Vec nor GloVe is used as Transformers are a newer class of algorithms. Word2Vec and GloVe are based on static word embeddings while Transformers are based on dynamic word embeddings. …
Brian O'Donnell's user avatar