New answers tagged transformer
0
To answer this, you need some constraints on the problem. Here are some sequences of numbers. No machine learning technique could be expected to learn all of them:
the odd numbers
the primes
numbers expressed in digits, but listed in alphabetical order of their name in German
numbers listed in the lexical order of the reverse of their representation in base ...
3
GPT-2 is a close copy of the basic transformer architecture.
GPT-2 does not require the encoder part of the original transformer architecture as it is decoder-only, and there are no encoder attention blocks, so the decoder is equivalent to the encoder, except for the MASKING in the multi-head attention block, the decoder is only allowed to glean information ...
0
There is an implementation of the paper ("Adversarial Sparse Transformer for Time Series Forecasting"), in Python using Pytorch, here. Although it has the training and evaluation functionality implemented, it appears to be lacking a function for running a prediction. Maybe you can fork it and extend it.
UPDATE
There is also a paper, "Informer:...
Top 50 recent answers are included
