New answers tagged performance
1
vote
Accepted
Is having many tokens generally good for NLP performance?
Indeed several studies provide evidence that the design of tokenizers significantly impacts transformer model performance such as the original BERT paper that introduced WordPiece tokenization ...
- 5,507
1
vote
Accepted
Transformer Encoder give worse performance when the sequence length during training is difference compared to inference
Indeed transformer with fixed-length positional encodings such as your SinePositionEncoding doesn't inherently generalize well to sequence lengths that differ from ...
- 5,507
Top 50 recent answers are included