Skip to main content

New answers tagged

1 vote
Accepted

Is having many tokens generally good for NLP performance?

Indeed several studies provide evidence that the design of tokenizers significantly impacts transformer model performance such as the original BERT paper that introduced WordPiece tokenization ...
cinch's user avatar
  • 5,422

Top 50 recent answers are included