1 vote

why we use learnable positional encoding instead of Sinusoidal positional encoding

transformers models use the sinusoidal position encoding to capture the position of each token, and this method has some limits : the sinusoidal position encoding is manually designed so it is not ...
LAILA EL OUEDEGHYRY's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible