New answers tagged hyperparameter-optimization
2
In most cases, seems that embedding dim is chosen empirically, by trial and error.
Older papers in NLP used 300 conventionally https://petuum.medium.com/embeddings-a-matrix-of-meaning-4de877c9aa27. More recent papers used 512, 768, 1024.
One of the factors, influencing the choice of embedding is the way you would like different vectors to correlate with each ...
1
I get an answer from this book: Machine Learning Design Patterns: Solutions to Common Challenges in Data Preparation, Model Building, and MLOps.
If we’re in a hurry, one rule of thumb is to use the fourth root of the total number of unique categorical elements while another is that the embedding dimension should be approximately 1.6 times the square root of ...
Top 50 recent answers are included
